09/14/2026 02:30:44.930 7f7bea458d40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:30:44.930 7f7bea458d40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:30:44.930 7f7bea458d40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:30:44.937 7f7bea458d40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:30:44.938 7f7be59ca2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:30:44.938 7f7be59ca2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:30:44.938 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:30:44.938 7f7be59ca7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:30:44.939 7f7be59ca7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:30:44.939 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:30:44.939 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.939 7f7be59ca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:30:44.939 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.939 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.939 7f7be59ca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353044939104 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:30:44.939 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.940 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.940 7f7be59ca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:30:44.940 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.940 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.940 7f7be59ca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:30:44.940 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.940 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.940 7f7be59ca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:30:44.940 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.940 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.941 7f7be59ca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:44.941 7f7be59ca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.941 7f7be597a2c0 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:30:44.941 7f7be597a2c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:30:44.942 7f7be59cae00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:30:44.943 7f7be597a9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.943 7f7be597a9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:30:44.943 7f7be597a9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.943 7f7be597a9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.943 7f7be597a9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:44.943 7f7be597a9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:44.996 7f7be59ca7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:30:45.004 7f7be59ca7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:30:45.005 7f7be5918040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.005 7f7be5918040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:45.005 7f7be5918040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.005 7f7be5918040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.006 7f7be5918040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:45.006 7f7be5918040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.007 7f7be5918040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.008 7f7be5918040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:45.008 7f7be5918040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.009 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:45.011 7f7be59ca7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:30:45.029 7f7be597ab80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.029 7f7be597ab80 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:45.030 7f7be597ab80 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.031 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:45.031 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:45.033 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:45.033 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:45.034 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:45.037 7f7be597ab80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.037 7f7be597ab80 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:45.037 7f7be597ab80 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.039 7f7be597ab80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.039 7f7be597ab80 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:45.039 7f7be597ab80 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.055 7f7be59ca7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:30:45.056 7f7be597ab80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.056 7f7be597ab80 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:45.056 7f7be597ab80 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.060 7f7be59ca7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:30:45.060 7f7be597ab80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.060 7f7be597ab80 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:45.060 7f7be597ab80 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.061 7f7be59ca7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:30:45.061 7f7be59ca7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:30:45.061 7f7be59ca7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:30:45.061 7f7be597ab80 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:30:45.062 7f7be58500e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.063 7f7be58500e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:45.063 7f7be58500e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.063 7f7be58500e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.063 7f7be58500e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:45.063 7f7be58500e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.066 7f7be59ca7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:30:45.069 7f7be597ab80 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:30:45.070 7f7be5850860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.070 7f7be5850860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:45.070 7f7be5850860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.107 7f7be59ca7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:30:45.116 7f7be6fa1400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.116 7f7be6fa1400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:30:45.117 7f7be6fa1400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:45.165 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:45.166 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:45.166 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:45.166 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:45.166 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:45.167 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:45.437 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:45.438 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:45.438 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:45.438 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:45.438 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:45.439 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:45.942 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:45.988 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:45.989 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:45.989 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:45.989 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:45.990 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:45.990 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:46.943 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:47.521 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:47.522 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:47.522 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:47.523 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:47.523 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:47.523 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:47.944 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:48.939 7f7be6fa1400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:48.940 7f7be6fa1400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:30:48.940 7f7be6fa1400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:48.941 7f7be6fa1400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:48.941 7f7be6fa1400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:30:48.941 7f7be6fa1400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:48.941 7f7be6fa1400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:30:48.941 7f7be6fa1400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:30:48.942 7f7be6fa1400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:30:48.942 7f7be6fa1400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:48.942 7f7be6fa1400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:48.942 7f7be6fa1900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:30:48.943 7f7be6fa1900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:30:48.946 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:49.655 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:49.656 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:49.656 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:49.656 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:49.657 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:49.657 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:49.947 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:50.948 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:51.949 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:52.950 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:53.951 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:54.874 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:54.875 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:54.875 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:54.875 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:54.875 7f7be597aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:54.876 7f7be597aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:54.952 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:55.954 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:56.959 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:57.960 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:58.961 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:59.962 7f7be597a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:00.026 7f7be597aea0 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:31:00.026 7f7be597aea0 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:31:00.027 7f7be85234a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:00.159 7f7be59ca7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:31:00.159 7f7bea458d40 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:31:00.159 7f7bea458d40 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:31:00.160 7f7be597aa40 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:31:00.160 7f7be597aa40 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:31:00.202 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:31:00.203 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:31:00.203 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:31:00.203 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:31:00.203 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:31:00.203 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:31:00.203 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:31:00.203 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:31:00.203 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:31:00.203 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:31:00.203 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:31:00.204 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:31:00.204 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:31:00.204 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:31:00.204 7f7bea458d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:31:09.454 7f66b5c85940 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:31:09.454 7f66b5c85940 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:31:09.455 7f66b5c85940 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:31:09.459 7f66b5c85940 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:31:09.459 7f66b0c362c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:31:09.459 7f66b0c362c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:31:09.460 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:31:09.460 7f66b0c367c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:31:09.461 7f66b0c367c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:31:09.461 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:31:09.461 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.461 7f66b0c367c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:31:09.461 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.461 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.461 7f66b0c367c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353069460995 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:31:09.461 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.462 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.462 7f66b0c367c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:09.462 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.462 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.463 7f66b0c367c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:09.463 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.463 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.463 7f66b0c367c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:09.463 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.463 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.463 7f66b0c367c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:09.463 7f66b0c367c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.464 7f66b0c36e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:09.464 7f66b0c36e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:31:09.465 7f66b0c36c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:31:09.465 7f66b0be6540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.466 7f66b0be6540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:09.466 7f66b0be6540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.466 7f66b0be6540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.466 7f66b0be6540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:09.466 7f66b0be6540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.520 7f66b0c367c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:31:09.527 7f66b0c367c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:31:09.528 7f66b0bcb040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.528 7f66b0bcb040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:09.528 7f66b0bcb040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.528 7f66b0bcb040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.528 7f66b0bcb040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:09.528 7f66b0bcb040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.530 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:09.531 7f66b0be67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.532 7f66b0be67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:09.532 7f66b0be67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.532 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:09.532 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:09.533 7f66b0c367c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:31:09.533 7f66b0be67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.533 7f66b0be67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:09.533 7f66b0be67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.539 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:09.539 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:09.541 7f66b0bcbd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.541 7f66b0bcbd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:09.541 7f66b0bcbd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.541 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:09.542 7f66b0bcbd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.543 7f66b0bcbd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:09.543 7f66b0bcbd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.564 7f66b0c367c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:09.565 7f66b0bcbc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.566 7f66b0bcbc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:09.566 7f66b0bcbc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.579 7f66b0c367c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:09.579 7f66b0bcbc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.579 7f66b0bcbc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:09.579 7f66b0bcbc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.579 7f66b0c367c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:31:09.580 7f66b0c367c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:31:09.580 7f66b0c367c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:31:09.580 7f66b0bcbf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:09.581 7f66b220a220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.581 7f66b220a220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:09.581 7f66b220a220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.581 7f66b220a220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.581 7f66b220a220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:09.581 7f66b220a220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.585 7f66b0c367c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:31:09.586 7f66b0bcbf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:09.586 7f66b220a9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.586 7f66b220a9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:09.586 7f66b220a9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.609 7f66b0c367c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:31:09.611 7f66b21e6400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.612 7f66b21e6400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:09.612 7f66b21e6400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:09.674 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:09.674 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:09.675 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:09.675 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:09.675 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:09.675 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:09.837 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:09.838 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:09.838 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:09.838 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:09.839 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:09.839 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:10.247 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:10.247 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:10.248 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:10.248 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:10.248 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:10.249 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:10.466 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:10.945 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:10.946 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:10.946 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:10.946 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:10.946 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:10.947 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:11.467 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:11.882 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:11.883 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:11.884 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:11.885 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:11.885 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:11.885 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:12.468 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:13.461 7f66b21e6400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:31:13.462 7f66b21e6400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:31:13.462 7f66b21e6400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:31:13.462 7f66b21e6400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:13.462 7f66b21e6400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:13.463 7f66b21e6400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:13.463 7f66b21e6400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:31:13.463 7f66b21e6400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:31:13.463 7f66b21e6400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:31:13.463 7f66b21e6400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:13.463 7f66b21e6400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:13.463 7f66b21e6900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:31:13.464 7f66b21e6900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:31:13.469 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:13.756 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:13.757 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:13.757 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:13.758 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:13.758 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:13.758 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:14.469 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:15.470 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:16.471 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:16.731 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:16.732 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:16.732 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:16.733 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:16.733 7f66b0be6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:16.733 7f66b0be6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:17.472 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:18.473 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:19.474 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:20.476 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:21.477 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:22.478 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:23.479 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:24.469 7f66b0be6a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:31:24.469 7f66b0be6a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:31:24.470 7f66b0be6a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:24.480 7f66b0c36f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:24.658 7f66b0c367c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:31:24.659 7f66b5c85940 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:31:24.659 7f66b5c85940 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:31:24.659 7f66b0be6680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:31:24.659 7f66b0be6680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:31:24.673 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:31:24.674 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:31:24.674 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:31:24.674 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:31:24.674 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:31:24.674 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:31:24.674 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:31:24.674 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:31:24.674 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:31:24.674 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:31:24.674 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:31:24.675 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:31:24.675 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:31:24.675 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:31:24.675 7f66b5c85940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:31:34.251 7f9033a32840 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:31:34.251 7f9033a32840 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:31:34.251 7f9033a32840 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:31:34.255 7f9033a32840 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:31:34.256 7f902ec752c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:31:34.256 7f902ec752c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:31:34.256 7f902ec757c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:31:34.257 7f902ec757c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:31:34.257 7f902ec757c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:31:34.257 7f902ec757c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:31:34.257 7f902ec757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.257 7f902ec757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:31:34.257 7f902ec757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.257 7f902ec757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.258 7f902ec757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353094257392 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:31:34.258 7f902ec757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.259 7f902ec757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.259 7f902ec757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:34.259 7f902ec757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.259 7f902ec757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.259 7f902ec757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:34.259 7f902ec757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.259 7f902ec757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.259 7f902ec757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:34.259 7f902ec757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.259 7f902ec757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.259 7f902ec757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:34.260 7f902ec757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.260 7f902ec75e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:34.260 7f902ec75e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:31:34.261 7f902ec75c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:31:34.262 7f902ec25540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.262 7f902ec25540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:34.262 7f902ec25540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.262 7f902ec25540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.262 7f902ec25540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:34.262 7f902ec25540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.324 7f902ec757c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:31:34.333 7f902ec757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:31:34.335 7f902ebc1040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.335 7f902ebc1040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:34.335 7f902ebc1040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.335 7f902ebc1040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.335 7f902ebc1040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:34.336 7f902ebc1040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.339 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:34.340 7f902ec257c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.341 7f902ec257c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:34.341 7f902ec257c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.342 7f902ec757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:31:34.342 7f902ec257c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.342 7f902ec257c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:34.343 7f902ec257c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.343 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:34.343 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:34.351 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:34.352 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:34.353 7f902ebc1d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.355 7f902ebc1d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:34.356 7f902ebc1d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.356 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:34.358 7f902ebc1d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.358 7f902ebc1d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:34.358 7f902ebc1d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.382 7f902ec757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:34.382 7f902ebc1c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.383 7f902ebc1c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:34.383 7f902ebc1c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.386 7f902ec757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:34.387 7f902ebc1c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.387 7f902ebc1c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:34.387 7f902ebc1c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.387 7f902ec757c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:31:34.388 7f902ec757c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:31:34.388 7f902ec757c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:31:34.388 7f902ebc1f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:34.389 7f902eaf1220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.390 7f902eaf1220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:34.390 7f902eaf1220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.390 7f902eaf1220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.390 7f902eaf1220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:34.390 7f902eaf1220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.394 7f902ebc1f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:34.395 7f902eaf14a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.395 7f902eaf14a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:34.395 7f902eaf14a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.399 7f902ec757c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:31:34.425 7f902ec757c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:31:34.426 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:34.427 7f902eaf1860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.428 7f902eaf1860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:34.428 7f902eaf1860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:34.428 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:34.428 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:34.429 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:34.429 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:34.430 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:34.553 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:34.553 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:34.553 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:34.554 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:34.554 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:34.555 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:34.864 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:34.865 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:34.865 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:34.865 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:34.865 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:34.866 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:35.261 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:35.394 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:35.395 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:35.395 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:35.395 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:35.396 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:35.396 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:36.263 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:36.276 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:36.276 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:36.276 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:36.277 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:36.277 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:36.277 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:37.264 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:37.345 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:37.346 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:37.346 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:37.346 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:37.346 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:37.347 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:38.275 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:38.275 7f902eaf1860 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:31:38.277 7f902eaf1860 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:31:38.277 7f902eaf1860 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:31:38.277 7f902eaf1860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:38.277 7f902eaf1860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:38.277 7f902eaf1860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:38.278 7f902eaf1860 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:31:38.278 7f902eaf1860 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:31:38.278 7f902eaf1860 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:31:38.278 7f902eaf1860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:38.279 7f902eaf1860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:38.279 7f90302fc180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:31:38.280 7f90302fc180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:31:39.276 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:40.084 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:40.084 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:40.084 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:40.085 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:40.085 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:40.086 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:40.277 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:41.278 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:42.280 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:43.281 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:44.283 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:44.835 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:44.836 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:44.836 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:44.837 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:44.837 7f902ec25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:44.837 7f902ec25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:45.284 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:46.285 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:47.286 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:48.287 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:49.288 7f902ec75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:49.359 7f902ec25a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:31:49.359 7f902ec25a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:31:49.360 7f902ec25a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:49.489 7f902ec757c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:31:49.490 7f9033a32840 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:31:49.490 7f9033a32840 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:31:49.490 7f902ec259a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:31:49.490 7f902ec259a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:31:49.504 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:31:49.505 7f9033a32840 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:31:58.728 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:31:58.728 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:31:58.728 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:31:58.731 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:31:58.732 7fd475aeb2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:31:58.732 7fd475aeb2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:31:58.732 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:31:58.732 7fd475aeb7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:31:58.733 7fd475aeb7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:31:58.733 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:31:58.733 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.733 7fd475aeb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:31:58.733 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.733 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.734 7fd475aeb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353118733253 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:31:58.734 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.734 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.735 7fd475aeb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:58.735 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.735 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.735 7fd475aeb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:58.735 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.735 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.735 7fd475aeb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:58.735 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.735 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.735 7fd475aeb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:58.735 7fd475aeb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.736 7fd475aebe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:58.736 7fd475aebe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:31:58.737 7fd475aebc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:31:58.737 7fd475a9b540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.738 7fd475a9b540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:58.738 7fd475a9b540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.738 7fd475a9b540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.738 7fd475a9b540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:58.738 7fd475a9b540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.795 7fd475aeb7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:31:58.802 7fd475aeb7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:31:58.803 7fd477bfb040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.803 7fd477bfb040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:58.804 7fd477bfb040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.804 7fd477bfb040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.804 7fd477bfb040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:58.804 7fd477bfb040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.807 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:58.807 7fd475a9b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.807 7fd475a9b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:58.807 7fd475a9b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.808 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:58.808 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:58.809 7fd475aeb7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:31:58.809 7fd475a9b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.810 7fd475a9b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:58.810 7fd475a9b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.810 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:58.810 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:58.811 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:58.817 7fd477bfbc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.818 7fd477bfbc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:58.818 7fd477bfbc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.819 7fd477bfbc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.819 7fd477bfbc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:58.819 7fd477bfbc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.858 7fd475aeb7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:58.858 7fd477bfbae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.859 7fd477bfbae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:58.859 7fd477bfbae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.862 7fd475aeb7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:58.862 7fd477bfbae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.862 7fd477bfbae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:58.862 7fd477bfbae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.863 7fd475aeb7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:31:58.863 7fd475aeb7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:31:58.863 7fd475aeb7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:31:58.863 7fd477bfbe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:58.864 7fd4770c20e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.864 7fd4770c20e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:58.864 7fd4770c20e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.864 7fd4770c20e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.864 7fd4770c20e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:58.864 7fd4770c20e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.868 7fd475aeb7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:31:58.873 7fd477bfbe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:58.873 7fd4770c2ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.874 7fd4770c2ea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:58.874 7fd4770c2ea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.899 7fd475aeb7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:31:58.901 7fd47709d400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.901 7fd47709d400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:58.901 7fd47709d400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:58.943 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:58.944 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:58.944 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:58.945 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:58.945 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:58.945 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:59.297 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:59.298 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:59.298 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:59.299 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:59.299 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:59.300 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:59.737 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:59.939 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:59.939 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:59.940 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:59.940 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:59.940 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:59.941 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:00.739 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:01.076 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:01.077 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:01.077 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:01.078 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:01.078 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:01.078 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:01.739 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:02.733 7fd4770c2720 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:32:02.735 7fd4770c2720 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:32:02.735 7fd4770c2720 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:32:02.735 7fd4770c2720 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:02.735 7fd4770c2720 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:02.735 7fd4770c2720 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:02.735 7fd4770c2720 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:32:02.735 7fd4770c2720 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:32:02.736 7fd4770c2720 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:32:02.736 7fd4770c2720 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:02.736 7fd4770c2720 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:02.736 7fd4770c2f40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:32:02.737 7fd4770c2f40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:32:02.740 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:02.993 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:02.993 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:02.994 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:02.994 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:02.994 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:02.995 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:03.742 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:04.743 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:05.390 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:05.391 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:05.392 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:05.392 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:05.392 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:05.393 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:05.744 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:06.747 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:07.747 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:08.749 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:09.750 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:10.751 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:11.511 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:11.512 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:11.512 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:11.512 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:11.512 7fd475a9ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:11.513 7fd475a9ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:11.752 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:12.753 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:13.754 7fd475aebf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:13.824 7fd475a9ba40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:32:13.824 7fd475a9ba40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:32:13.825 7fd475a9ba40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:13.909 7fd475aeb7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:32:13.909 7fd47a8a98c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:32:13.910 7fd47a8a98c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:32:13.910 7fd475a9b860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:32:13.910 7fd475a9b860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:32:13.924 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:32:13.924 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:32:13.924 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:32:13.924 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:32:13.925 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:32:13.925 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:32:13.925 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:32:13.925 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:32:13.925 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:32:13.925 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:32:13.925 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:32:13.925 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:32:13.925 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:32:13.925 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:32:13.925 7fd47a8a98c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:32:22.957 7fd23790ab00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:32:22.957 7fd23790ab00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:32:22.958 7fd23790ab00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:32:22.961 7fd23790ab00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:32:22.961 7fd2328c02c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:32:22.961 7fd2328c02c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:32:22.962 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:32:22.962 7fd2328c07c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:32:22.962 7fd2328c07c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:32:22.963 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:32:22.963 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.963 7fd2328c07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:32:22.963 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.963 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.963 7fd2328c07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353142962782 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:32:22.963 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.964 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.964 7fd2328c07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:22.964 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.964 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.964 7fd2328c07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:22.964 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.964 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.965 7fd2328c07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:22.965 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.965 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.965 7fd2328c07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:22.965 7fd2328c07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.965 7fd2328c0e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:22.965 7fd2328c0e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:32:22.966 7fd2328c0c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:32:22.967 7fd232870540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.967 7fd232870540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:22.967 7fd232870540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.967 7fd232870540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:22.967 7fd232870540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:22.967 7fd232870540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.028 7fd2328c07c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:32:23.037 7fd2328c07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:32:23.037 7fd232809040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.038 7fd232809040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:23.038 7fd232809040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.038 7fd232809040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.038 7fd232809040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:23.038 7fd232809040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.039 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:23.040 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:23.040 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:23.041 7fd2328707c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.041 7fd2328707c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:23.041 7fd2328707c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.042 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:23.043 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:23.043 7fd2328c07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:32:23.043 7fd2328707c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.044 7fd2328707c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:23.044 7fd2328707c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.051 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:23.052 7fd232809c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.052 7fd232809c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:23.052 7fd232809c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.054 7fd232809c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.054 7fd232809c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:23.054 7fd232809c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.077 7fd2328c07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:23.078 7fd232809ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.078 7fd232809ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:23.078 7fd232809ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.082 7fd2328c07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:23.082 7fd232809ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.082 7fd232809ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:23.082 7fd232809ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.083 7fd2328c07c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:32:23.083 7fd2328c07c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:32:23.083 7fd2328c07c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:32:23.083 7fd232809e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:23.084 7fd2327420e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.084 7fd2327420e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:23.084 7fd2327420e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.084 7fd2327420e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.084 7fd2327420e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:23.084 7fd2327420e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.089 7fd2328c07c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:32:23.091 7fd232809e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:23.092 7fd2327429a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.092 7fd2327429a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:23.092 7fd2327429a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.117 7fd2328c07c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:32:23.119 7fd233f53400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.119 7fd233f53400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:23.119 7fd233f53400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:23.147 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:23.148 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:23.148 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:23.148 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:23.148 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:23.149 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:23.379 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:23.380 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:23.380 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:23.380 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:23.380 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:23.381 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:23.886 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:23.887 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:23.887 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:23.887 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:23.887 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:23.888 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:23.967 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:24.726 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:24.727 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:24.727 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:24.728 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:24.728 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:24.728 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:24.968 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:25.970 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:26.260 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:26.261 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:26.261 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:26.262 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:26.262 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:26.262 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:26.963 7fd233f53400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:32:26.964 7fd233f53400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:32:26.964 7fd233f53400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:32:26.965 7fd233f53400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:26.965 7fd233f53400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:26.965 7fd233f53400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:26.965 7fd233f53400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:32:26.965 7fd233f53400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:32:26.966 7fd233f53400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:32:26.966 7fd233f53400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:26.966 7fd233f53400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:26.966 7fd233f53900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:32:26.967 7fd233f53900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:32:26.971 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:27.972 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:28.973 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:29.975 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:30.194 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:30.195 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:30.195 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:30.196 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:30.196 7fd232870a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:30.196 7fd232870a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:30.976 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:31.977 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:32.978 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:33.979 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:34.981 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:35.982 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:36.984 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:37.984 7fd2328c0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:38.032 7fd232870a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:32:38.033 7fd232870a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:32:38.033 7fd232870a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:38.193 7fd2328c07c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:32:38.193 7fd23790ab00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:32:38.193 7fd23790ab00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:32:38.194 7fd232870860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:32:38.194 7fd232870860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:32:38.207 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:32:38.208 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:32:38.208 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:32:38.208 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:32:38.208 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:32:38.208 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:32:38.208 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:32:38.208 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:32:38.208 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:32:38.208 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:32:38.208 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:32:38.208 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:32:38.209 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:32:38.209 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:32:38.209 7fd23790ab00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:32:48.277 7f0eef164d40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:32:48.277 7f0eef164d40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:32:48.277 7f0eef164d40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:32:48.280 7f0eef164d40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:32:48.281 7f0eea39c2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:32:48.281 7f0eea39c2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:32:48.281 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:32:48.282 7f0eea39c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:32:48.282 7f0eea39c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:32:48.282 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:32:48.282 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.282 7f0eea39c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:32:48.283 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.283 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.283 7f0eea39c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353168282505 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:32:48.283 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.284 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.284 7f0eea39c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:48.284 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.284 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.284 7f0eea39c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:48.284 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.284 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.284 7f0eea39c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:48.284 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.284 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.284 7f0eea39c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:48.284 7f0eea39c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.285 7f0eea39ce00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:48.285 7f0eea39ce00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:32:48.286 7f0eea39cc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:32:48.286 7f0eea34c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.287 7f0eea34c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:48.287 7f0eea34c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.287 7f0eea34c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.287 7f0eea34c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:48.287 7f0eea34c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.353 7f0eea39c7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:32:48.362 7f0eea39c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:32:48.363 7f0eea2ef040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.363 7f0eea2ef040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:48.363 7f0eea2ef040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.363 7f0eea2ef040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.363 7f0eea2ef040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:48.363 7f0eea2ef040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.365 7f0eea34c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.365 7f0eea34c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:48.365 7f0eea34c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.366 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:48.367 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.367 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:48.368 7f0eea39c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:32:48.368 7f0eea34c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.368 7f0eea34c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:48.368 7f0eea34c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.375 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.375 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:48.376 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.377 7f0eea2efc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.377 7f0eea2efc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:48.377 7f0eea2efc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.379 7f0eea2efc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.379 7f0eea2efc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:48.379 7f0eea2efc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.395 7f0eea39c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:48.396 7f0eea2efae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.396 7f0eea2efae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:48.396 7f0eea2efae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.399 7f0eea39c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:48.399 7f0eea2efae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.399 7f0eea2efae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:48.400 7f0eea2efae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.400 7f0eea39c7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:32:48.400 7f0eea39c7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:32:48.401 7f0eea39c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:32:48.401 7f0eea2efe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:48.401 7f0eea2200e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.402 7f0eea2200e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:48.402 7f0eea2200e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.402 7f0eea2200e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.402 7f0eea2200e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:48.403 7f0eea2200e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.407 7f0eea39c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:32:48.408 7f0eea2efe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:48.408 7f0eea2209a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.409 7f0eea2209a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:48.409 7f0eea2209a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.432 7f0eea39c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:32:48.433 7f0eeba32180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.434 7f0eeba32180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:48.434 7f0eeba32180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:48.443 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:48.443 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.444 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:48.444 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.444 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:48.445 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.629 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:48.630 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.630 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:48.630 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.630 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:48.631 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.968 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:48.968 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.968 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:48.970 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.970 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:48.970 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:49.286 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:49.503 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:49.504 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:49.504 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:49.505 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:49.505 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:49.505 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:50.227 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:50.228 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:50.228 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:50.229 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:50.229 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:50.229 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:50.287 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:51.288 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:51.894 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:51.895 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:51.895 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:51.895 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:51.895 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:51.896 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:52.282 7f0eea2207c0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:32:52.283 7f0eea2207c0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:32:52.283 7f0eea2207c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:32:52.283 7f0eea2207c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:52.283 7f0eea2207c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:52.284 7f0eea2207c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:52.284 7f0eea2207c0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:32:52.284 7f0eea2207c0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:32:52.284 7f0eea2207c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:32:52.284 7f0eea2207c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:52.284 7f0eea2207c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:52.284 7f0eeba32680 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:32:52.285 7f0eeba32680 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:32:52.289 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:53.290 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:54.291 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:54.510 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:54.511 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:54.511 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:54.511 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:54.512 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:54.512 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:55.293 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:56.294 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:57.295 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:58.296 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:59.297 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:00.299 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:01.158 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:01.159 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:01.159 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:01.160 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:01.160 7f0eea34ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:01.160 7f0eea34ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:01.300 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:02.302 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:03.303 7f0eea39cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:03.370 7f0eea34ca40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:33:03.370 7f0eea34ca40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:33:03.371 7f0eea34ca40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:03.500 7f0eea39c7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:33:03.500 7f0eef164d40 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:33:03.500 7f0eef164d40 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:33:03.501 7f0eea34c860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:33:03.501 7f0eea34c860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:33:03.515 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:33:03.515 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:33:03.516 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:33:03.516 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:33:03.516 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:33:03.516 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:33:03.516 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:33:03.516 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:33:03.516 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:33:03.516 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:33:03.516 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:33:03.516 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:33:03.516 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:33:03.516 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:33:03.516 7f0eef164d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:33:12.489 7fd61e16ac80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:33:12.489 7fd61e16ac80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:33:12.490 7fd61e16ac80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:33:12.493 7fd61e16ac80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:33:12.493 7fd61911e2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:33:12.494 7fd61911e2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:33:12.494 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:33:12.494 7fd61911e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:33:12.495 7fd61911e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:33:12.495 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:33:12.495 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.495 7fd61911e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:33:12.495 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.495 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.495 7fd61911e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353192494962 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:33:12.495 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.496 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.496 7fd61911e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:12.496 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.496 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.496 7fd61911e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:12.496 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.496 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.497 7fd61911e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:12.497 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.497 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.497 7fd61911e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:12.497 7fd61911e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.497 7fd61911ee00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:12.497 7fd61911ee00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:33:12.498 7fd61911ec20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:33:12.499 7fd6190cd540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.499 7fd6190cd540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:12.499 7fd6190cd540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.499 7fd6190cd540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.499 7fd6190cd540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:12.499 7fd6190cd540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.556 7fd61911e7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:33:12.564 7fd61911e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:33:12.565 7fd61906d040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.565 7fd61906d040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:12.566 7fd61906d040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.566 7fd61906d040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.566 7fd61906d040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:12.566 7fd61906d040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.568 7fd6190cd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.569 7fd6190cd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:12.569 7fd6190cd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.570 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:12.571 7fd61911e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:33:12.572 7fd61906d040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.572 7fd61906d040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:12.572 7fd61906d040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.572 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:12.572 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:12.573 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:12.573 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:12.580 7fd61906dd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.580 7fd61906dd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:12.580 7fd61906dd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.581 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:12.583 7fd61906dd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.583 7fd61906dd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:12.583 7fd61906dd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.607 7fd61911e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:12.608 7fd61906dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.608 7fd61906dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:12.608 7fd61906dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.611 7fd61911e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:12.612 7fd61906dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.612 7fd61906dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:12.612 7fd61906dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.612 7fd61911e7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:33:12.612 7fd61911e7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:33:12.613 7fd61911e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:33:12.613 7fd61906df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:12.613 7fd61a6cc220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.614 7fd61a6cc220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:12.614 7fd61a6cc220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.614 7fd61a6cc220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.614 7fd61a6cc220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:12.614 7fd61a6cc220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.618 7fd61911e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:33:12.619 7fd61906df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:12.620 7fd61a6cc9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.620 7fd61a6cc9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:12.620 7fd61a6cc9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.657 7fd61911e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:33:12.659 7fd61a6cc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.659 7fd61a6cc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:33:12.659 7fd61a6cc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:12.673 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:12.673 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:12.674 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:12.674 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:12.674 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:12.675 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:12.801 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:12.801 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:12.801 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:12.802 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:12.802 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:12.802 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:13.067 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:13.068 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:13.068 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:13.068 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:13.068 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:13.069 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:13.499 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:13.582 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:13.583 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:13.583 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:13.584 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:13.584 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:13.584 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:14.499 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:14.500 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:14.500 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:14.500 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:14.501 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:14.501 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:14.502 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:15.501 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:16.024 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:16.025 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:16.025 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:16.025 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:16.025 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:16.026 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:16.495 7fd61a6cc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:33:16.496 7fd61a6cc7c0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:33:16.496 7fd61a6cc7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:33:16.496 7fd61a6cc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:16.497 7fd61a6cc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:33:16.497 7fd61a6cc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:16.497 7fd61a6cc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:33:16.497 7fd61a6cc7c0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:33:16.497 7fd61a6cc7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:33:16.497 7fd61a6cc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:16.497 7fd61a6cc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:16.498 7fd61a6f8180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:33:16.498 7fd61a6f8180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:33:16.502 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:17.504 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:18.443 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:18.444 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:18.444 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:18.445 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:18.445 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:18.445 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:18.505 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:19.506 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:20.507 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:21.509 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:22.510 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:22.829 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:22.830 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:22.830 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:22.832 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:22.833 7fd6190cda40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:22.833 7fd6190cda40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:23.511 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:24.513 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:25.514 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:26.515 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:27.516 7fd61911ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:27.556 7fd6190cda40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:33:27.557 7fd6190cda40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:33:27.557 7fd6190cda40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:27.745 7fd61911e7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:33:27.745 7fd61e16ac80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:33:27.745 7fd61e16ac80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:33:27.746 7fd6190cd9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:33:27.746 7fd6190cd9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:33:27.760 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:33:27.760 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:33:27.760 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:33:27.760 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:33:27.760 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:33:27.760 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:33:27.760 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:33:27.760 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:33:27.760 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:33:27.760 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:33:27.760 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:33:27.760 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:33:27.761 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:33:27.761 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:33:27.761 7fd61e16ac80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:33:37.069 7f04632711c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:33:37.069 7f04632711c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:33:37.069 7f04632711c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:33:37.072 7f04632711c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:33:37.073 7f045e0a22c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:33:37.073 7f045e0a22c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:33:37.074 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:33:37.074 7f045e0a27c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:33:37.074 7f045e0a27c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:33:37.074 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:33:37.075 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.075 7f045e0a27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:33:37.075 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.075 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.075 7f045e0a27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353217074641 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:33:37.075 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.076 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.076 7f045e0a27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:37.076 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.076 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.076 7f045e0a27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:37.076 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.076 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.076 7f045e0a27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:37.076 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.077 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.077 7f045e0a27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:37.077 7f045e0a27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.077 7f045e0a2e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:37.077 7f045e0a2e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:33:37.078 7f045e0a2c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:33:37.079 7f045e052540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.079 7f045e052540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:37.079 7f045e052540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.079 7f045e052540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.079 7f045e052540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:37.079 7f045e052540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.132 7f045e0a27c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:33:37.139 7f045e0a27c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:33:37.140 7f045dfad040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.140 7f045dfad040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:37.140 7f045dfad040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.141 7f045dfad040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.141 7f045dfad040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:37.141 7f045dfad040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.142 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:37.143 7f045e0527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.143 7f045e0527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:37.143 7f045e0527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.144 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.144 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:37.145 7f045e0a27c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:33:37.145 7f045e0527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.145 7f045e0527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:37.145 7f045e0527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.146 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.146 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:37.152 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.153 7f045dfadc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.153 7f045dfadc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:37.153 7f045dfadc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.155 7f045dfadc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.155 7f045dfadc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:37.155 7f045dfadc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.198 7f045e0a27c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:37.199 7f045dfadae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.199 7f045dfadae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:37.199 7f045dfadae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.203 7f045e0a27c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:37.204 7f045dfadae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.204 7f045dfadae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:37.204 7f045dfadae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.204 7f045e0a27c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:33:37.204 7f045e0a27c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:33:37.205 7f045e0a27c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:33:37.205 7f045dfade00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:37.205 7f045df260e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.206 7f045df260e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:37.206 7f045df260e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.206 7f045df260e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.206 7f045df260e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:37.206 7f045df260e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.210 7f045e0a27c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:33:37.212 7f045dfade00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:37.212 7f045df269a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.212 7f045df269a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:37.212 7f045df269a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.245 7f045e0a27c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:33:37.247 7f045df267c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.247 7f045df267c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:33:37.247 7f045df267c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:37.283 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:37.284 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.284 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:37.285 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.285 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:37.286 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.480 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:37.480 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.480 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:37.481 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.481 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:37.482 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.864 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:37.865 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.866 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:37.866 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.866 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:37.867 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:38.078 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:38.559 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:38.560 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:38.560 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:38.561 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:38.561 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:38.561 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:39.080 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:40.081 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:40.146 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:40.147 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:40.147 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:40.147 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:40.147 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:40.148 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:41.075 7f045df267c0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:33:41.076 7f045df267c0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:33:41.076 7f045df267c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:33:41.076 7f045df267c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:41.076 7f045df267c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:33:41.077 7f045df267c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:41.077 7f045df267c0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:33:41.077 7f045df267c0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:33:41.077 7f045df267c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:33:41.077 7f045df267c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:41.077 7f045df267c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:41.077 7f045f728180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:33:41.078 7f045f728180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:33:41.082 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:42.083 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:43.084 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:43.955 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:43.956 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:43.956 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:43.956 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:43.956 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:43.957 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:44.086 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:45.087 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:46.088 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:47.090 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:48.090 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:49.092 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:50.093 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:51.094 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:51.684 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:51.685 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:51.685 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:51.685 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:51.685 7f045e052a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:51.686 7f045e052a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:52.088 7f045e052a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:33:52.088 7f045e052a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:33:52.089 7f045e052a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:52.095 7f045e0a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:52.266 7f045e0a27c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:33:52.266 7f04632711c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:33:52.267 7f04632711c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:33:52.267 7f045e052860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:33:52.267 7f045e052860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:33:52.280 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:33:52.280 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:33:52.280 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:33:52.280 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:33:52.280 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:33:52.280 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:33:52.280 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:33:52.280 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:33:52.281 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:33:52.281 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:33:52.281 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:33:52.281 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:33:52.281 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:33:52.281 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:33:52.281 7f04632711c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:34:01.570 7f0accfebf00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:34:01.570 7f0accfebf00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:34:01.570 7f0accfebf00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:34:01.575 7f0accfebf00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:34:01.576 7f0ac85522c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:34:01.576 7f0ac85522c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:34:01.578 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:34:01.578 7f0ac85527c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:34:01.579 7f0ac85527c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:34:01.580 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:34:01.580 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.580 7f0ac85527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:34:01.580 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.580 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.580 7f0ac85527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353241578983 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:34:01.580 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.581 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.582 7f0ac85527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:01.582 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.582 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.582 7f0ac85527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:01.582 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.582 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.582 7f0ac85527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:01.582 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.582 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.582 7f0ac85527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:01.583 7f0ac85527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.583 7f0ac8552e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:01.583 7f0ac8552e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:34:01.585 7f0ac8552c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:34:01.585 7f0ac8502540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.585 7f0ac8502540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:01.586 7f0ac8502540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.586 7f0ac8502540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.586 7f0ac8502540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:01.586 7f0ac8502540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.684 7f0ac85527c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:34:01.695 7f0ac85527c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:34:01.696 7f0ac849e040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.697 7f0ac849e040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:01.697 7f0ac849e040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.697 7f0ac849e040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.697 7f0ac849e040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:01.697 7f0ac849e040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.699 7f0ac849e040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.735 7f0ac849e040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:01.735 7f0ac849e040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.737 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:01.738 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:01.738 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:01.739 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:01.739 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:01.740 7f0ac85027c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.740 7f0ac85027c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:01.740 7f0ac85027c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.742 7f0ac85527c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:34:01.744 7f0ac85027c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.744 7f0ac85027c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:01.744 7f0ac85027c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.744 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:01.764 7f0ac849ec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.765 7f0ac849ec20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:01.765 7f0ac849ec20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.807 7f0ac85527c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:01.808 7f0ac849eae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.808 7f0ac849eae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:01.809 7f0ac849eae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.813 7f0ac85527c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:01.814 7f0ac849eae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.814 7f0ac849eae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:01.814 7f0ac849eae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.815 7f0ac85527c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:34:01.815 7f0ac85527c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:34:01.816 7f0ac85527c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:34:01.816 7f0ac849ee00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:01.819 7f0ac83d00e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.821 7f0ac83d00e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:01.821 7f0ac83d00e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.821 7f0ac83d00e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.821 7f0ac83d00e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:01.821 7f0ac83d00e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.832 7f0ac849ee00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:01.838 7f0ac83d00e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.842 7f0ac83d00e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:01.843 7f0ac83d00e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.843 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:01.855 7f0ac85527c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:34:01.860 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:01.861 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:01.864 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:01.867 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:01.875 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:01.954 7f0ac85527c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:34:01.961 7f0ac9be7400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.963 7f0ac9be7400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:01.963 7f0ac9be7400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:01.967 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:01.974 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:01.975 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:01.975 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:01.976 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:01.977 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:02.147 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:02.147 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:02.148 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:02.148 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:02.149 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:02.149 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:02.515 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:02.516 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:02.516 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:02.517 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:02.517 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:02.517 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:02.585 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:03.099 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:03.100 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:03.100 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:03.101 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:03.101 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:03.101 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:03.586 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:03.828 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:03.829 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:03.829 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:03.829 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:03.829 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:03.830 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:04.588 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:05.579 7f0ac9be7400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:34:05.580 7f0ac9be7400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:34:05.581 7f0ac9be7400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:34:05.581 7f0ac9be7400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:05.581 7f0ac9be7400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:05.581 7f0ac9be7400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:05.581 7f0ac9be7400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:34:05.581 7f0ac9be7400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:34:05.582 7f0ac9be7400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:34:05.582 7f0ac9be7400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:05.583 7f0ac9be7400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:05.583 7f0ac9be7900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:34:05.584 7f0ac9be7900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:34:05.588 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:05.673 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:05.674 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:05.674 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:05.674 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:05.675 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:05.675 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:06.589 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:07.590 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:08.591 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:09.592 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:10.002 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:10.003 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:10.003 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:10.004 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:10.004 7f0ac8502a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:10.004 7f0ac8502a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:10.593 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:11.594 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:12.596 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:13.597 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:14.598 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:15.600 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:16.601 7f0ac8552f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:16.645 7f0ac8502a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:34:16.645 7f0ac8502a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:34:16.646 7f0ac8502a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:16.991 7f0ac85527c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:34:16.991 7f0accfebf00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:34:16.991 7f0accfebf00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:34:16.991 7f0ac85029a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:34:16.992 7f0ac85029a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:34:17.006 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:34:17.006 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:34:17.006 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:34:17.006 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:34:17.006 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:34:17.006 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:34:17.006 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:34:17.007 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:34:17.007 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:34:17.007 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:34:17.007 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:34:17.007 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:34:17.007 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:34:17.007 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:34:17.007 7f0accfebf00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:34:25.972 7fc495777dc0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:34:25.973 7fc495777dc0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:34:25.973 7fc495777dc0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:34:25.977 7fc495777dc0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:34:25.977 7fc490ce92c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:34:25.977 7fc490ce92c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:34:25.977 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:34:25.978 7fc490ce97c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:34:25.978 7fc490ce97c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:34:25.978 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:34:25.978 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.978 7fc490ce97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:34:25.979 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.979 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.979 7fc490ce97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353265978588 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:34:25.979 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.979 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.979 7fc490ce97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:25.980 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.980 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.980 7fc490ce97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:25.980 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.980 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.980 7fc490ce97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:25.980 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.980 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.980 7fc490ce97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:25.980 7fc490ce97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.980 7fc490ce9e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:25.981 7fc490ce9e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:34:25.981 7fc490ce9c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:34:25.982 7fc490c99540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.982 7fc490c99540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:25.982 7fc490c99540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.982 7fc490c99540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:25.982 7fc490c99540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:25.982 7fc490c99540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.041 7fc490ce97c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:34:26.049 7fc490ce97c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:34:26.050 7fc492dfa040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.051 7fc492dfa040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:26.051 7fc492dfa040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.051 7fc492dfa040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.051 7fc492dfa040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:26.051 7fc492dfa040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.052 7fc492dfa040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.053 7fc492dfa040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:26.053 7fc492dfa040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.053 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:26.054 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:26.055 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:26.055 7fc490ce97c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:34:26.056 7fc490c997c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.056 7fc490c997c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:26.056 7fc490c997c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.056 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:26.057 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:26.058 7fc492dfa040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.058 7fc492dfa040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:26.058 7fc492dfa040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.058 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:26.066 7fc492dfac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.067 7fc492dfac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:26.067 7fc492dfac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.094 7fc490ce97c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:26.095 7fc492dfaae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.095 7fc492dfaae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:26.095 7fc492dfaae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.098 7fc490ce97c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:26.099 7fc492dfaae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.099 7fc492dfaae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:26.099 7fc492dfaae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.099 7fc490ce97c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:34:26.099 7fc490ce97c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:34:26.099 7fc490ce97c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:34:26.100 7fc492dfae00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:26.101 7fc4922c10e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.101 7fc4922c10e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:26.101 7fc4922c10e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.101 7fc4922c10e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.101 7fc4922c10e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:26.101 7fc4922c10e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.105 7fc490ce97c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:34:26.107 7fc492dfae00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:26.107 7fc4922c1860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.107 7fc4922c1860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:26.107 7fc4922c1860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.129 7fc490ce97c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:34:26.131 7fc49229c180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.131 7fc49229c180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:26.131 7fc49229c180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:26.169 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:26.170 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:26.170 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:26.170 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:26.170 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:26.171 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:26.383 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:26.384 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:26.384 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:26.384 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:26.384 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:26.385 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:26.757 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:26.758 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:26.758 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:26.758 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:26.759 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:26.759 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:26.981 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:27.746 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:27.747 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:27.747 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:27.748 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:27.748 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:27.748 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:27.982 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:28.984 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:29.903 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:29.904 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:29.904 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:29.905 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:29.905 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:29.906 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:29.978 7fc49229c680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:34:29.979 7fc49229c680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:34:29.979 7fc49229c680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:34:29.979 7fc49229c680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:29.980 7fc49229c680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:29.980 7fc49229c680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:29.980 7fc49229c680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:34:29.980 7fc49229c680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:34:29.980 7fc49229c680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:34:29.980 7fc49229c680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:29.980 7fc49229c680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:29.980 7fc49229c900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:34:29.981 7fc49229c900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:34:29.985 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:30.986 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:31.987 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:32.988 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:33.989 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:34.991 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:35.873 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:35.874 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:35.874 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:35.874 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:35.874 7fc490c99a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:35.875 7fc490c99a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:35.992 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:36.992 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:37.993 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:38.995 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:39.996 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:40.997 7fc490ce9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:41.000 7fc490c99a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:34:41.000 7fc490c99a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:34:41.001 7fc490c99a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:41.176 7fc490ce97c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:34:41.176 7fc495777dc0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:34:41.176 7fc495777dc0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:34:41.177 7fc490c99860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:34:41.177 7fc490c99860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:34:41.190 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:34:41.190 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:34:41.190 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:34:41.190 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:34:41.190 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:34:41.190 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:34:41.190 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:34:41.190 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:34:41.190 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:34:41.190 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:34:41.191 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:34:41.191 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:34:41.191 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:34:41.191 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:34:41.191 7fc495777dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:34:49.902 7fb902e5cac0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:34:49.902 7fb902e5cac0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:34:49.902 7fb902e5cac0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:34:49.906 7fb902e5cac0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:34:49.906 7fb8fe3cb2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:34:49.906 7fb8fe3cb2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:34:49.906 7fb8fe3cb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:34:49.907 7fb8fe3cb7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:34:49.907 7fb8fe3cb7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:34:49.907 7fb8fe3cb7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:34:49.907 7fb8fe3cb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.907 7fb8fe3cb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:34:49.908 7fb8fe3cb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.908 7fb8fe3cb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.908 7fb8fe3cb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353289907443 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:34:49.908 7fb8fe3cb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.909 7fb8fe3cbc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.909 7fb8fe3cbc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:49.909 7fb8fe3cbc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.909 7fb8fe3cbc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.909 7fb8fe3cbc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:49.909 7fb8fe3cbc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.909 7fb8fe3cbc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.909 7fb8fe3cbc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:49.909 7fb8fe3cbc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.909 7fb8fe3cbc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.910 7fb8fe3cbc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:49.910 7fb8fe3cbc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.910 7fb8fe3cbf40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:49.910 7fb8fe3cbf40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:34:49.911 7fb8fe3cbe00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:34:49.911 7fb8fe37b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.912 7fb8fe37b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:49.912 7fb8fe37b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.912 7fb8fe37b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.912 7fb8fe37b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:49.912 7fb8fe37b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.965 7fb8fe3cbc20 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:34:49.973 7fb8fe3cbc20 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:34:49.974 7fb9004d70e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.974 7fb9004d70e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:49.974 7fb9004d70e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.974 7fb9004d70e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.974 7fb9004d70e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:49.974 7fb9004d70e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.977 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:49.978 7fb8fe37b9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.978 7fb8fe37b9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:49.979 7fb8fe37b9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.979 7fb8fe3cbc20 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:34:49.980 7fb8fe37b9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.980 7fb8fe37b9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:49.980 7fb8fe37b9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.980 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:49.980 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:49.987 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:49.987 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:49.988 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:49.989 7fb9004d7d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.989 7fb9004d7d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:49.990 7fb9004d7d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.994 7fb9004d7d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:49.994 7fb9004d7d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:49.994 7fb9004d7d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.021 7fb8fe3cbc20 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:50.022 7fb9004d7c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.022 7fb9004d7c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:50.022 7fb9004d7c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.025 7fb8fe3cbc20 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:50.025 7fb9004d7c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.026 7fb9004d7c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:50.026 7fb9004d7c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.026 7fb8fe3cbc20 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:34:50.026 7fb8fe3cbc20 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:34:50.026 7fb8fe3cbc20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:34:50.027 7fb9004d7f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:50.027 7fb8ff9a0220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.028 7fb8ff9a0220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:50.028 7fb8ff9a0220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.028 7fb8ff9a0220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.028 7fb8ff9a0220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:50.028 7fb8ff9a0220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.031 7fb8fe3cbc20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:34:50.033 7fb9004d7f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:50.033 7fb8ff9a09a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.033 7fb8ff9a09a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:50.033 7fb8ff9a09a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.062 7fb8fe3cbc20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:34:50.064 7fb8ff97d180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.064 7fb8ff97d180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:50.064 7fb8ff97d180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.128 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:50.128 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:50.128 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:50.129 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:50.129 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:50.129 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:50.300 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:50.301 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:50.301 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:50.302 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:50.302 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:50.302 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:50.592 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:50.592 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:50.593 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:50.593 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:50.593 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:50.594 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:50.911 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:51.239 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:51.240 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:51.240 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:51.240 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:51.240 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:51.241 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:51.912 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:52.583 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:52.584 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:52.584 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:52.584 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:52.585 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:52.585 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:52.913 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:53.908 7fb8ff97d680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:34:53.909 7fb8ff97d680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:34:53.909 7fb8ff97d680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:34:53.909 7fb8ff97d680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:53.910 7fb8ff97d680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:53.910 7fb8ff97d680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:53.910 7fb8ff97d680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:34:53.910 7fb8ff97d680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:34:53.910 7fb8ff97d680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:34:53.910 7fb8ff97d680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:53.910 7fb8ff97d680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:53.911 7fb8ff97d900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:34:53.912 7fb8ff97d900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:34:53.915 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:54.823 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:54.824 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:54.824 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:54.824 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:54.824 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:54.825 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:54.916 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:55.917 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:56.918 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:57.919 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:57.971 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:57.972 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:57.972 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:57.973 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:57.973 7fb8fe37bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:57.973 7fb8fe37bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:58.920 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:59.921 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:00.922 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:01.923 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:02.923 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:03.924 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:04.925 7fb8fe37b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:05.011 7fb8fe37bc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:35:05.011 7fb8fe37bc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:35:05.011 7fb8fe37bc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:05.120 7fb8fe3cbc20 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:35:05.120 7fb902e5cac0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:35:05.121 7fb902e5cac0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:35:05.121 7fb8fe37b860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:35:05.121 7fb8fe37b860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:35:05.152 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:35:05.152 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:35:05.152 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:35:05.152 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:35:05.152 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:35:05.152 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:35:05.152 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:35:05.152 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:35:05.152 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:35:05.153 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:35:05.153 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:35:05.153 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:35:05.153 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:35:05.153 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:35:05.153 7fb902e5cac0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:35:14.041 7f24d55926c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:35:14.042 7f24d55926c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:35:14.042 7f24d55926c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:35:14.045 7f24d55926c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:35:14.045 7f24d0aff2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:35:14.046 7f24d0aff2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:35:14.046 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:35:14.046 7f24d0aff7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:35:14.047 7f24d0aff7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:35:14.047 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:35:14.047 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.047 7f24d0aff7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:35:14.047 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.047 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.047 7f24d0aff7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353314047070 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:35:14.047 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.048 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.048 7f24d0aff7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:14.049 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.049 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.049 7f24d0aff7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:14.049 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.049 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.049 7f24d0aff7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:14.049 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.049 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.049 7f24d0aff7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:14.049 7f24d0aff7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.050 7f24d0affe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:14.050 7f24d0affe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:35:14.051 7f24d0affc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:35:14.052 7f24d0aaf540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.052 7f24d0aaf540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:14.052 7f24d0aaf540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.052 7f24d0aaf540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.052 7f24d0aaf540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:14.052 7f24d0aaf540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.123 7f24d0aff7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:35:14.131 7f24d0aff7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:35:14.133 7f24d0a4a040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.133 7f24d0a4a040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:14.133 7f24d0a4a040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.133 7f24d0a4a040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.133 7f24d0a4a040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:14.133 7f24d0a4a040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.135 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:14.136 7f24d0aaf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.136 7f24d0aaf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:14.137 7f24d0aaf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.138 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:14.138 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:14.140 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:14.140 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:14.140 7f24d0aaf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.141 7f24d0aaf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:14.141 7f24d0aaf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.141 7f24d0aff7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:35:14.142 7f24d0aaf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.142 7f24d0aaf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:14.142 7f24d0aaf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.142 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:14.154 7f24d0a4ad60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.154 7f24d0a4ad60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:14.154 7f24d0a4ad60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.183 7f24d0aff7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:14.184 7f24d0a4ac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.184 7f24d0a4ac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:14.184 7f24d0a4ac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.188 7f24d0aff7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:14.188 7f24d0a4ac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.188 7f24d0a4ac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:14.188 7f24d0a4ac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.188 7f24d0aff7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:35:14.189 7f24d0aff7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:35:14.189 7f24d0aff7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:35:14.189 7f24d0a4af40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:14.191 7f24d0988220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.191 7f24d0988220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:14.191 7f24d0988220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.191 7f24d0988220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.191 7f24d0988220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:14.191 7f24d0988220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.195 7f24d0aff7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:35:14.197 7f24d0a4af40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:14.197 7f24d09889a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.197 7f24d09889a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:14.198 7f24d09889a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.229 7f24d0aff7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:35:14.230 7f24d2192400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.231 7f24d2192400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:35:14.231 7f24d2192400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:14.281 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:14.281 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:14.282 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:14.282 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:14.283 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:14.283 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:14.521 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:14.522 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:14.522 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:14.522 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:14.523 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:14.523 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:15.046 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:15.047 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:15.047 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:15.047 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:15.047 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:15.050 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:15.051 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:16.052 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:16.109 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:16.110 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:16.110 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:16.111 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:16.111 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:16.111 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:17.054 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:17.404 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:17.405 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:17.405 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:17.406 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:17.408 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:17.408 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:18.047 7f24d2192400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:35:18.048 7f24d2192400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:35:18.049 7f24d2192400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:35:18.049 7f24d2192400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:18.049 7f24d2192400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:35:18.049 7f24d2192400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:18.049 7f24d2192400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:35:18.049 7f24d2192400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:35:18.050 7f24d2192400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:35:18.050 7f24d2192400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:18.050 7f24d2192400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:18.050 7f24d2192900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:35:18.051 7f24d2192900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:35:18.055 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:19.056 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:19.464 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:19.465 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:19.465 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:19.465 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:19.466 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:19.466 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:20.056 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:21.057 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:22.058 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:22.827 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:22.828 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:22.828 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:22.829 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:22.829 7f24d0aafa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:22.829 7f24d0aafa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:23.059 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:24.061 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:25.062 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:26.063 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:27.063 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:28.065 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:29.060 7f24d0aafa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:35:29.060 7f24d0aafa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:35:29.061 7f24d0aafa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:29.066 7f24d0afff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:29.243 7f24d0aff7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:35:29.243 7f24d55926c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:35:29.243 7f24d55926c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:35:29.243 7f24d0aaf680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:35:29.244 7f24d0aaf680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:35:29.259 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:35:29.260 7f24d55926c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:35:38.309 7fefb47bb600 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:35:38.309 7fefb47bb600 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:35:38.309 7fefb47bb600 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:35:38.313 7fefb47bb600 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:35:38.314 7fefafd352c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:35:38.314 7fefafd352c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:35:38.314 7fefafd357c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:35:38.314 7fefafd357c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:35:38.315 7fefafd357c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:35:38.315 7fefafd357c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:35:38.315 7fefafd357c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.315 7fefafd357c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:35:38.315 7fefafd357c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.315 7fefafd357c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.315 7fefafd357c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353338314986 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:35:38.315 7fefafd357c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.316 7fefafd357c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.316 7fefafd357c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:38.316 7fefafd357c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.316 7fefafd357c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.316 7fefafd357c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:38.316 7fefafd357c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.316 7fefafd357c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.316 7fefafd357c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:38.317 7fefafd357c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.317 7fefafd357c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.317 7fefafd357c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:38.317 7fefafd357c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.317 7fefafd35e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:38.317 7fefafd35e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:35:38.318 7fefafd35c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:35:38.319 7fefafce4540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.319 7fefafce4540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:38.319 7fefafce4540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.319 7fefafce4540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.319 7fefafce4540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:38.319 7fefafce4540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.374 7fefafd357c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:35:38.381 7fefafd357c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:35:38.382 7fefafcc2040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.383 7fefafcc2040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:38.383 7fefafcc2040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.383 7fefafcc2040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.383 7fefafcc2040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:38.383 7fefafcc2040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.384 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:38.386 7fefafce47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.386 7fefafce47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:38.386 7fefafce47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.387 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:38.387 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:38.389 7fefafd357c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:35:38.389 7fefafce47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.389 7fefafce47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:38.389 7fefafce47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.390 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:38.390 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:38.390 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:38.398 7fefafcc2c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.399 7fefafcc2c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:38.399 7fefafcc2c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.403 7fefafcc2c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.404 7fefafcc2c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:38.404 7fefafcc2c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.433 7fefafd357c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:38.434 7fefafcc2ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.434 7fefafcc2ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:38.434 7fefafcc2ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.437 7fefafd357c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:38.437 7fefafcc2ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.438 7fefafcc2ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:38.438 7fefafcc2ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.438 7fefafd357c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:35:38.438 7fefafd357c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:35:38.438 7fefafd357c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:35:38.439 7fefafcc2e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:38.439 7fefafbb80e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.440 7fefafbb80e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:38.440 7fefafbb80e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.440 7fefafbb80e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.440 7fefafbb80e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:38.440 7fefafbb80e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.449 7fefafd357c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:35:38.455 7fefafcc2e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:38.456 7fefafbb8ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.456 7fefafbb8ea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:38.456 7fefafbb8ea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.477 7fefafd357c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:35:38.479 7fefb13ca540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.480 7fefb13ca540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:35:38.480 7fefb13ca540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:38.480 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:38.486 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:38.486 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:38.487 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:38.487 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:38.488 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:38.606 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:38.607 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:38.608 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:38.609 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:38.609 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:38.609 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:38.757 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:38.758 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:38.758 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:38.758 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:38.759 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:38.759 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:39.143 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:39.143 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:39.144 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:39.144 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:39.144 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:39.145 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:39.318 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:39.842 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:39.843 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:39.843 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:39.844 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:39.844 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:39.844 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:40.320 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:41.096 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:41.097 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:41.098 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:41.098 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:41.098 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:41.099 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:41.321 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:42.315 7fefb13ca540 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:35:42.316 7fefb13ca540 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:35:42.316 7fefb13ca540 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:35:42.316 7fefb13ca540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:42.316 7fefb13ca540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:35:42.316 7fefb13ca540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:42.316 7fefb13ca540 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:35:42.317 7fefb13ca540 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:35:42.317 7fefb13ca540 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:35:42.317 7fefb13ca540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:42.317 7fefb13ca540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:42.317 7fefb13caa40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:35:42.318 7fefb13caa40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:35:42.322 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:43.166 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:43.167 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:43.167 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:43.168 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:43.168 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:43.168 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:43.323 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:44.324 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:45.325 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:45.936 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:45.936 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:45.936 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:45.937 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:45.937 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:45.938 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:46.326 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:47.327 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:48.329 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:49.330 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:50.331 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:51.332 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:51.674 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:51.675 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:51.675 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:51.675 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:51.675 7fefafce4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:51.676 7fefafce4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:52.333 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:53.334 7fefafd35f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:53.383 7fefafce4a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:35:53.383 7fefafce4a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:35:53.384 7fefafce4a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:53.580 7fefafd357c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:35:53.580 7fefb47bb600 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:35:53.580 7fefb47bb600 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:35:53.580 7fefafce4860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:35:53.581 7fefafce4860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:35:53.634 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:35:53.658 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:35:53.658 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:35:53.658 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:35:53.658 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:35:53.658 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:35:53.658 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:35:53.658 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:35:53.659 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:35:53.659 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:35:53.659 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:35:53.659 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:35:53.659 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:35:53.659 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:35:53.659 7fefb47bb600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:36:03.134 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:36:03.134 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:36:03.134 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:36:03.138 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:36:03.138 7f1b1bfcd2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:36:03.138 7f1b1bfcd2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:36:03.139 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:36:03.139 7f1b1bfcd7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:36:03.139 7f1b1bfcd7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:36:03.140 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:36:03.140 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.140 7f1b1bfcd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:36:03.140 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.140 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.140 7f1b1bfcd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353363139730 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:36:03.140 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.141 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.141 7f1b1bfcd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:03.141 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.141 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.141 7f1b1bfcd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:03.141 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.141 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.141 7f1b1bfcd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:03.142 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.142 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.142 7f1b1bfcd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:03.142 7f1b1bfcd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.142 7f1b1bfcde00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:03.142 7f1b1bfcde00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:36:03.143 7f1b1bfcdc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:36:03.144 7f1b1bf7d540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.144 7f1b1bf7d540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:03.144 7f1b1bf7d540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.144 7f1b1bf7d540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.144 7f1b1bf7d540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:03.144 7f1b1bf7d540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.206 7f1b1bfcd7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:36:03.214 7f1b1bfcd7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:36:03.215 7f1b1e0de040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.215 7f1b1e0de040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:03.215 7f1b1e0de040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.215 7f1b1e0de040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.215 7f1b1e0de040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:03.216 7f1b1e0de040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.217 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:03.218 7f1b1bf7d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.218 7f1b1bf7d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:03.219 7f1b1bf7d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.219 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:03.220 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:03.220 7f1b1bfcd7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:36:03.221 7f1b1bf7d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.221 7f1b1bf7d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:03.221 7f1b1bf7d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.221 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:03.221 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:03.228 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:03.230 7f1b1e0ded60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.230 7f1b1e0ded60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:03.230 7f1b1e0ded60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.232 7f1b1e0ded60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.232 7f1b1e0ded60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:03.232 7f1b1e0ded60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.256 7f1b1bfcd7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:03.257 7f1b1e0dec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.257 7f1b1e0dec20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:03.257 7f1b1e0dec20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.260 7f1b1bfcd7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:03.260 7f1b1e0dec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.261 7f1b1e0dec20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:03.261 7f1b1e0dec20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.261 7f1b1bfcd7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:36:03.261 7f1b1bfcd7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:36:03.261 7f1b1bfcd7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:36:03.262 7f1b1e0def40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:03.263 7f1b1d5a3220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.263 7f1b1d5a3220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:03.263 7f1b1d5a3220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.263 7f1b1d5a3220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.263 7f1b1d5a3220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:03.263 7f1b1d5a3220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.267 7f1b1bfcd7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:36:03.269 7f1b1e0def40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:03.269 7f1b1d5a39a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.269 7f1b1d5a39a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:03.269 7f1b1d5a39a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.299 7f1b1bfcd7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:36:03.302 7f1b1d5a35e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.302 7f1b1d5a35e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:03.302 7f1b1d5a35e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:03.335 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:03.335 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:03.335 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:03.336 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:03.336 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:03.336 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:03.577 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:03.578 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:03.578 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:03.578 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:03.579 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:03.579 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:04.091 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:04.092 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:04.092 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:04.092 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:04.092 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:04.093 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:04.143 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:05.144 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:05.367 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:05.368 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:05.368 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:05.368 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:05.368 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:05.369 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:06.145 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:07.140 7f1b1d5a35e0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:36:07.141 7f1b1d5a35e0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:36:07.142 7f1b1d5a35e0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:36:07.142 7f1b1d5a35e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:07.142 7f1b1d5a35e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:07.142 7f1b1d5a35e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:07.142 7f1b1d5a35e0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:36:07.142 7f1b1d5a35e0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:36:07.142 7f1b1d5a35e0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:36:07.142 7f1b1d5a35e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:07.142 7f1b1d5a35e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:07.143 7f1b1d58d180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:36:07.144 7f1b1d58d180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:36:07.146 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:07.383 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:07.384 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:07.384 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:07.384 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:07.385 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:07.385 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:08.147 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:09.148 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:09.888 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:09.889 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:09.889 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:09.890 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:09.890 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:09.890 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:10.149 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:11.150 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:12.151 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:13.152 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:14.153 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:15.155 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:15.177 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:15.178 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:15.178 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:15.178 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:15.179 7f1b1bf7da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:15.179 7f1b1bf7da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:16.156 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:17.157 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:18.158 7f1b1bfcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:18.192 7f1b1bf7da40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:36:18.193 7f1b1bf7da40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:36:18.193 7f1b1bf7da40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:18.380 7f1b1bfcd7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:36:18.380 7f1b20d9cf00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:36:18.380 7f1b20d9cf00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:36:18.380 7f1b1bf7d9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:36:18.381 7f1b1bf7d9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:36:18.400 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:36:18.400 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:36:18.400 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:36:18.400 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:36:18.400 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:36:18.400 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:36:18.400 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:36:18.400 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:36:18.400 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:36:18.401 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:36:18.401 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:36:18.401 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:36:18.401 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:36:18.401 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:36:18.401 7f1b20d9cf00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:36:27.281 7fefce182800 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:36:27.282 7fefce182800 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:36:27.282 7fefce182800 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:36:27.285 7fefce182800 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:36:27.286 7fefc96fe2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:36:27.286 7fefc96fe2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:36:27.286 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:36:27.287 7fefc96fe7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:36:27.287 7fefc96fe7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:36:27.287 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:36:27.287 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.287 7fefc96fe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:36:27.287 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.288 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.288 7fefc96fe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353387287316 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:36:27.288 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.289 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.289 7fefc96fe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:27.289 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.289 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.289 7fefc96fe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:27.289 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.289 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.289 7fefc96fe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:27.290 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.290 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.290 7fefc96fe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:27.290 7fefc96fe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.290 7fefc96fee00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:27.290 7fefc96fee00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:36:27.291 7fefc96fec20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:36:27.292 7fefc96ae540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.292 7fefc96ae540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:27.292 7fefc96ae540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.292 7fefc96ae540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.293 7fefc96ae540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:27.293 7fefc96ae540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.355 7fefc96fe7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:36:27.364 7fefc96fe7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:36:27.365 7fefc96050e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.365 7fefc96050e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:27.365 7fefc96050e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.365 7fefc96050e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.365 7fefc96050e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:27.365 7fefc96050e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.367 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:27.368 7fefc96ae7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.368 7fefc96ae7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:27.368 7fefc96ae7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.369 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:27.369 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:27.370 7fefc96fe7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:36:27.370 7fefc96ae7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.371 7fefc96ae7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:27.371 7fefc96ae7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.379 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:27.379 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:27.380 7fefc9605d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.381 7fefc9605d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:27.381 7fefc9605d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.381 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:27.384 7fefc9605d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.384 7fefc9605d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:27.384 7fefc9605d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.413 7fefc96fe7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:27.414 7fefc9605c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.414 7fefc9605c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:27.415 7fefc9605c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.418 7fefc96fe7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:27.418 7fefc9605c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.419 7fefc9605c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:27.419 7fefc9605c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.419 7fefc96fe7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:36:27.419 7fefc96fe7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:36:27.420 7fefc96fe7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:36:27.420 7fefc9605f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:27.421 7fefc9585220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.422 7fefc9585220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:27.422 7fefc9585220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.422 7fefc9585220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.422 7fefc9585220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:27.422 7fefc9585220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.427 7fefc96fe7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:36:27.429 7fefc9605f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:27.430 7fefc95859a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.430 7fefc95859a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:27.430 7fefc95859a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.467 7fefc96fe7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:36:27.470 7fefcad91400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.471 7fefcad91400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:27.471 7fefcad91400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:27.475 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:27.476 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:27.477 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:27.477 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:27.477 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:27.478 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:27.655 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:27.678 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:27.678 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:27.679 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:27.679 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:27.680 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:27.981 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:27.981 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:27.982 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:27.982 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:27.982 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:27.983 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:28.291 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:28.409 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:28.410 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:28.410 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:28.411 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:28.411 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:28.411 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:29.292 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:29.483 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:29.484 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:29.484 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:29.485 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:29.485 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:29.486 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:30.293 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:31.287 7fefcad91400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:36:31.288 7fefcad91400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:36:31.288 7fefcad91400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:36:31.288 7fefcad91400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:31.289 7fefcad91400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:31.289 7fefcad91400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:31.289 7fefcad91400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:36:31.289 7fefcad91400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:36:31.289 7fefcad91400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:36:31.289 7fefcad91400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:31.289 7fefcad91400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:31.290 7fefcad91900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:36:31.291 7fefcad91900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:36:31.294 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:31.405 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:31.405 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:31.406 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:31.406 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:31.406 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:31.407 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:32.295 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:33.298 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:34.298 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:34.695 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:34.696 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:34.696 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:34.697 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:34.697 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:34.697 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:35.300 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:36.301 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:37.302 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:38.304 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:39.229 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:39.230 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:39.230 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:39.231 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:39.231 7fefc96aec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:39.231 7fefc96aec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:39.305 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:40.318 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:41.319 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:42.320 7fefc96fef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:42.345 7fefc96aec20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:36:42.346 7fefc96aec20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:36:42.346 7fefc96aec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:42.522 7fefc96fe7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:36:42.523 7fefce182800 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:36:42.523 7fefce182800 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:36:42.523 7fefc96ae680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:36:42.523 7fefc96ae680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:36:42.537 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:36:42.537 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:36:42.537 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:36:42.537 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:36:42.537 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:36:42.537 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:36:42.538 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:36:42.538 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:36:42.538 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:36:42.538 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:36:42.538 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:36:42.538 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:36:42.538 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:36:42.538 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:36:42.538 7fefce182800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:36:51.513 7f25b3a17c80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:36:51.513 7f25b3a17c80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:36:51.513 7f25b3a17c80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:36:51.516 7f25b3a17c80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:36:51.517 7f25aef992c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:36:51.517 7f25aef992c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:36:51.517 7f25aef997c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:36:51.517 7f25aef997c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:36:51.518 7f25aef997c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:36:51.518 7f25aef997c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:36:51.518 7f25aef997c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.518 7f25aef997c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:36:51.518 7f25aef997c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.518 7f25aef997c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.518 7f25aef997c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353411518042 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:36:51.518 7f25aef997c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.519 7f25aef997c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.519 7f25aef997c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:51.519 7f25aef997c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.519 7f25aef997c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.520 7f25aef997c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:51.520 7f25aef997c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.520 7f25aef997c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.520 7f25aef997c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:51.520 7f25aef997c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.520 7f25aef997c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.520 7f25aef997c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:51.520 7f25aef997c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.520 7f25aef99e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:51.520 7f25aef99e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:36:51.521 7f25aef99c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:36:51.522 7f25aef48540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.522 7f25aef48540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:51.522 7f25aef48540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.522 7f25aef48540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.522 7f25aef48540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:51.522 7f25aef48540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.598 7f25aef997c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:36:51.609 7f25aef997c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:36:51.613 7f25aeee7040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.613 7f25aeee7040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:51.613 7f25aeee7040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.613 7f25aeee7040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.613 7f25aeee7040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:51.613 7f25aeee7040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.616 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:51.618 7f25aef997c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:36:51.626 7f25aef487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.626 7f25aef487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:51.626 7f25aef487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.626 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:51.627 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:51.627 7f25aef487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.628 7f25aef487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:51.628 7f25aef487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.628 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:51.628 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:51.630 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:51.632 7f25aef487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.633 7f25aef487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:51.633 7f25aef487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.634 7f25aef487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.635 7f25aef487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:51.635 7f25aef487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.668 7f25aef997c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:51.669 7f25aef487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.669 7f25aef487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:51.670 7f25aef487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.673 7f25aef997c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:51.674 7f25aef487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.674 7f25aef487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:51.674 7f25aef487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.675 7f25aef997c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:36:51.675 7f25aef997c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:36:51.675 7f25aef997c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:36:51.676 7f25aef487c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:51.676 7f25aee200e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.677 7f25aee200e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:51.677 7f25aee200e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.677 7f25aee200e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.677 7f25aee200e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:51.677 7f25aee200e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.683 7f25aef997c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:36:51.685 7f25aef487c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:51.686 7f25aee20400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.686 7f25aee20400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:51.686 7f25aee20400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.720 7f25aef997c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:36:51.722 7f25b056c400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.722 7f25b056c400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:51.722 7f25b056c400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:51.742 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:51.743 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:51.743 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:51.743 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:51.743 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:51.744 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:51.984 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:51.985 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:51.985 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:51.986 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:51.986 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:51.986 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:52.399 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:52.400 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:52.400 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:52.401 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:52.401 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:52.402 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:52.522 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:53.523 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:53.555 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:53.556 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:53.556 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:53.557 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:53.557 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:53.557 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:54.524 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:55.061 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:55.062 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:55.062 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:55.063 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:55.063 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:55.063 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:55.518 7f25b056c400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:36:55.519 7f25b056c400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:36:55.520 7f25b056c400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:36:55.520 7f25b056c400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:55.520 7f25b056c400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:55.520 7f25b056c400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:55.520 7f25b056c400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:36:55.520 7f25b056c400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:36:55.520 7f25b056c400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:36:55.520 7f25b056c400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:55.520 7f25b056c400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:55.521 7f25b056c900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:36:55.521 7f25b056c900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:36:55.525 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:56.526 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:57.527 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:58.528 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:58.707 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:58.708 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:58.708 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:58.708 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:58.709 7f25aef48a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:58.709 7f25aef48a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:59.529 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:00.530 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:01.532 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:02.533 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:03.535 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:04.536 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:05.537 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:06.539 7f25aef99f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:06.553 7f25aef48a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:37:06.553 7f25aef48a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:37:06.554 7f25aef48a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:06.768 7f25aef997c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:37:06.769 7f25b3a17c80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:37:06.769 7f25b3a17c80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:37:06.769 7f25aef48860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:37:06.769 7f25aef48860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:37:06.784 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:37:06.784 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:37:06.784 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:37:06.784 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:37:06.784 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:37:06.784 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:37:06.784 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:37:06.785 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:37:06.785 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:37:06.785 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:37:06.785 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:37:06.785 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:37:06.785 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:37:06.785 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:37:06.785 7f25b3a17c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:37:16.051 7f60b98d3580 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:37:16.051 7f60b98d3580 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:37:16.051 7f60b98d3580 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:37:16.055 7f60b98d3580 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:37:16.055 7f60b4e4e2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:37:16.055 7f60b4e4e2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:37:16.056 7f60b4e4e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:37:16.056 7f60b4e4e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:37:16.056 7f60b4e4e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:37:16.057 7f60b4e4e7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:37:16.057 7f60b4e4e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.057 7f60b4e4e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:37:16.057 7f60b4e4e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.057 7f60b4e4e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.057 7f60b4e4e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353436056813 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:37:16.057 7f60b4e4e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.058 7f60b4e4e900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.058 7f60b4e4e900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:16.058 7f60b4e4e900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.058 7f60b4e4e900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.058 7f60b4e4e900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:16.058 7f60b4e4e900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.059 7f60b4e4e900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.059 7f60b4e4e900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:16.059 7f60b4e4e900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.059 7f60b4e4e900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.059 7f60b4e4e900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:16.059 7f60b4e4e900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.060 7f60b4e4ef40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:16.060 7f60b4e4ef40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:37:16.061 7f60b4e4ee00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:37:16.062 7f60b4dfe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.062 7f60b4dfe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:16.062 7f60b4dfe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.062 7f60b4dfe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.062 7f60b4dfe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:16.062 7f60b4dfe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.125 7f60b4e4e900 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:37:16.134 7f60b4e4e900 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:37:16.135 7f60b4d5d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.136 7f60b4d5d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:16.136 7f60b4d5d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.136 7f60b4d5d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.136 7f60b4d5d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:16.136 7f60b4d5d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.137 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:16.139 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:16.139 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:16.140 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:16.140 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:16.141 7f60b4e4e900 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:37:16.141 7f60b4dfe9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.141 7f60b4dfe9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:16.142 7f60b4dfe9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.143 7f60b4dfe9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.143 7f60b4dfe9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:16.143 7f60b4dfe9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.143 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:16.153 7f60b4d5dd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.153 7f60b4d5dd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:16.154 7f60b4d5dd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.156 7f60b4d5dd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.156 7f60b4d5dd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:16.156 7f60b4d5dd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.190 7f60b4e4e900 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:16.191 7f60b4d5dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.191 7f60b4d5dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:16.191 7f60b4d5dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.196 7f60b4e4e900 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:16.196 7f60b4d5dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.196 7f60b4d5dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:16.196 7f60b4d5dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.197 7f60b4e4e900 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:37:16.197 7f60b4e4e900 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:37:16.197 7f60b4e4e900 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:37:16.198 7f60b4d5df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:16.198 7f60b4cd1220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.198 7f60b4cd1220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:16.198 7f60b4cd1220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.198 7f60b4cd1220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.199 7f60b4cd1220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:16.199 7f60b4cd1220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.203 7f60b4e4e900 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:37:16.207 7f60b4d5df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:16.208 7f60b4cd19a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.208 7f60b4cd19a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:16.208 7f60b4cd19a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.228 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:16.230 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:16.230 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:16.231 7f60b4e4e900 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:37:16.231 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:16.231 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:16.233 7f60b64e1400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.233 7f60b64e1400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:37:16.233 7f60b64e1400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.234 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:16.428 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:16.429 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:16.429 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:16.430 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:16.430 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:16.430 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:16.717 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:16.718 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:16.718 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:16.718 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:16.718 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:16.719 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:17.062 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:17.260 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:17.260 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:17.261 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:17.261 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:17.261 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:17.261 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:18.063 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:18.651 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:18.652 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:18.652 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:18.653 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:18.653 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:18.654 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:19.064 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:20.057 7f60b64e1400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:37:20.058 7f60b64e1400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:37:20.058 7f60b64e1400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:37:20.058 7f60b64e1400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:20.058 7f60b64e1400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:37:20.058 7f60b64e1400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:20.058 7f60b64e1400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:37:20.058 7f60b64e1400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:37:20.059 7f60b64e1400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:37:20.059 7f60b64e1400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:20.059 7f60b64e1400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:20.059 7f60b64e1900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:37:20.060 7f60b64e1900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:37:20.065 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:21.066 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:21.903 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:21.904 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:21.904 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:21.904 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:21.905 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:21.905 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:22.067 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:23.068 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:24.069 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:25.071 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:26.072 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:27.073 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:28.074 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:29.076 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:29.776 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:29.777 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:29.777 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:29.778 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:29.778 7f60b4dfec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:29.778 7f60b4dfec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:30.077 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:31.078 7f60b4dfe2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:31.085 7f60b4dfec20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:37:31.085 7f60b4dfec20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:37:31.086 7f60b4dfec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:31.251 7f60b4e4e900 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:37:31.251 7f60b98d3580 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:37:31.252 7f60b98d3580 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:37:31.252 7f60b4dfe860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:37:31.252 7f60b4dfe860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:37:31.265 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:37:31.266 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:37:31.266 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:37:31.266 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:37:31.266 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:37:31.266 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:37:31.266 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:37:31.266 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:37:31.266 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:37:31.266 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:37:31.266 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:37:31.266 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:37:31.266 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:37:31.267 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:37:31.267 7f60b98d3580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:37:40.164 7f1000ea8900 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:37:40.164 7f1000ea8900 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:37:40.164 7f1000ea8900 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:37:40.168 7f1000ea8900 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:37:40.168 7f0ffbe582c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:37:40.168 7f0ffbe582c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:37:40.168 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:37:40.169 7f0ffbe587c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:37:40.169 7f0ffbe587c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:37:40.169 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:37:40.169 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.169 7f0ffbe587c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:37:40.169 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.170 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.170 7f0ffbe587c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353460169495 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:37:40.170 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.170 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.171 7f0ffbe587c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:40.171 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.171 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.171 7f0ffbe587c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:40.171 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.171 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.171 7f0ffbe587c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:40.171 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.171 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.171 7f0ffbe587c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:40.171 7f0ffbe587c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.172 7f0ffbe58e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:40.172 7f0ffbe58e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:37:40.173 7f0ffbe58c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:37:40.173 7f0ffbe08540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.173 7f0ffbe08540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:40.173 7f0ffbe08540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.173 7f0ffbe08540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.173 7f0ffbe08540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:40.174 7f0ffbe08540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.230 7f0ffbe587c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:37:40.238 7f0ffbe587c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:37:40.239 7f0ffbde9040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.239 7f0ffbde9040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:40.239 7f0ffbde9040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.239 7f0ffbde9040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.239 7f0ffbde9040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:40.239 7f0ffbde9040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.240 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:40.242 7f0ffbe087c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.242 7f0ffbe087c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:40.242 7f0ffbe087c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.242 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:40.243 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:40.243 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:40.244 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:40.244 7f0ffbe587c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:37:40.245 7f0ffbe087c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.245 7f0ffbe087c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:40.245 7f0ffbe087c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.253 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:40.254 7f0ffbde9c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.255 7f0ffbde9c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:40.255 7f0ffbde9c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.257 7f0ffbde9c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.257 7f0ffbde9c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:40.257 7f0ffbde9c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.279 7f0ffbe587c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:40.280 7f0ffbde9ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.280 7f0ffbde9ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:40.281 7f0ffbde9ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.283 7f0ffbe587c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:40.284 7f0ffbde9ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.284 7f0ffbde9ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:40.284 7f0ffbde9ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.284 7f0ffbe587c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:37:40.284 7f0ffbe587c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:37:40.284 7f0ffbe587c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:37:40.285 7f0ffbde9e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:40.285 7f0ffb7270e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.286 7f0ffb7270e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:40.286 7f0ffb7270e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.286 7f0ffb7270e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.286 7f0ffb7270e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:40.286 7f0ffb7270e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.290 7f0ffbe587c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:37:40.292 7f0ffbde9e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:40.292 7f0ffb7279a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.293 7f0ffb7279a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:40.293 7f0ffb7279a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.320 7f0ffbe587c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:37:40.323 7f0ffb7277c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.323 7f0ffb7277c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:37:40.323 7f0ffb7277c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.325 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:40.325 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:40.326 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:40.326 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:40.326 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:40.327 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:40.441 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:40.441 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:40.441 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:40.442 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:40.442 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:40.442 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:40.731 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:40.731 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:40.732 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:40.732 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:40.732 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:40.733 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:41.173 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:41.215 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:41.216 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:41.216 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:41.216 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:41.216 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:41.217 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:42.173 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:42.417 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:42.417 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:42.418 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:42.418 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:42.418 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:42.419 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:43.175 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:44.033 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:44.034 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:44.034 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:44.034 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:44.035 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:44.035 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:44.169 7f0ffb7277c0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:37:44.171 7f0ffb7277c0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:37:44.171 7f0ffb7277c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:37:44.171 7f0ffb7277c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:44.171 7f0ffb7277c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:37:44.171 7f0ffb7277c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:44.171 7f0ffb7277c0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:37:44.171 7f0ffb7277c0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:37:44.172 7f0ffb7277c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:37:44.172 7f0ffb7277c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:44.172 7f0ffb7277c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:44.172 7f0ffdf58180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:37:44.173 7f0ffdf58180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:37:44.176 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:45.176 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:46.177 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:47.179 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:48.124 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:48.125 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:48.125 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:48.126 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:48.126 7f0ffbe08a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:48.127 7f0ffbe08a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:48.180 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:49.181 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:50.182 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:51.183 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:52.184 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:53.185 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:54.187 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:55.188 7f0ffbe58f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:55.271 7f0ffbe08a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:37:55.271 7f0ffbe08a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:37:55.272 7f0ffbe08a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:55.369 7f0ffbe587c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:37:55.369 7f1000ea8900 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:37:55.369 7f1000ea8900 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:37:55.369 7f0ffbe08860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:37:55.369 7f0ffbe08860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:37:55.382 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:37:55.383 7f1000ea8900 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:38:05.237 7fc667980d80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:38:05.237 7fc667980d80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:38:05.237 7fc667980d80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:38:05.241 7fc667980d80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:38:05.241 7fc662eed2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:38:05.241 7fc662eed2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:38:05.242 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:38:05.242 7fc662eed7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:38:05.242 7fc662eed7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:38:05.242 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:38:05.243 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.243 7fc662eed7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:38:05.243 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.243 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.243 7fc662eed7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353485242601 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:38:05.243 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.244 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.244 7fc662eed7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:05.244 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.244 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.244 7fc662eed7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:05.245 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.245 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.245 7fc662eed7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:05.245 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.245 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.245 7fc662eed7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:05.245 7fc662eed7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.245 7fc662eede00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:05.246 7fc662eede00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:38:05.247 7fc662eedc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:38:05.247 7fc662e9d540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.247 7fc662e9d540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:05.247 7fc662e9d540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.247 7fc662e9d540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.248 7fc662e9d540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:05.248 7fc662e9d540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.305 7fc662eed7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:38:05.314 7fc662eed7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:38:05.314 7fc662e84040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.315 7fc662e84040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:05.315 7fc662e84040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.315 7fc662e84040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.315 7fc662e84040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:05.315 7fc662e84040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.317 7fc662e84860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.317 7fc662e84860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:05.317 7fc662e84860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.318 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:05.318 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:05.318 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:05.320 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:05.320 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:05.320 7fc662eed7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:38:05.321 7fc662e9d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.321 7fc662e9d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:05.321 7fc662e9d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.321 7fc662e9d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.321 7fc662e9d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:05.321 7fc662e9d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.328 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:05.329 7fc662e84d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.329 7fc662e84d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:05.329 7fc662e84d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.366 7fc662eed7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:05.367 7fc662e84c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.367 7fc662e84c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:05.367 7fc662e84c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.371 7fc662eed7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:05.372 7fc662e84c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.372 7fc662e84c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:05.372 7fc662e84c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.372 7fc662eed7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:38:05.372 7fc662eed7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:38:05.372 7fc662eed7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:38:05.373 7fc662e84f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:05.373 7fc6627bf220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.374 7fc6627bf220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:05.374 7fc6627bf220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.374 7fc6627bf220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.374 7fc6627bf220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:05.374 7fc6627bf220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.377 7fc662eed7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:38:05.378 7fc662e84f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:05.378 7fc6627bf9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.379 7fc6627bf9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:05.379 7fc6627bf9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.403 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:05.404 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:05.404 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:05.405 7fc662eed7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:38:05.405 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:05.405 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:05.407 7fc664ff6180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.408 7fc664ff6180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:05.408 7fc664ff6180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:05.408 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:05.600 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:05.601 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:05.601 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:05.602 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:05.602 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:05.602 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:06.021 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:06.022 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:06.022 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:06.023 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:06.023 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:06.023 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:06.246 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:07.086 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:07.087 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:07.087 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:07.087 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:07.087 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:07.088 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:07.248 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:08.248 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:09.243 7fc664ff6680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:38:09.244 7fc664ff6680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:38:09.244 7fc664ff6680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:38:09.244 7fc664ff6680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:09.244 7fc664ff6680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:09.244 7fc664ff6680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:09.245 7fc664ff6680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:38:09.245 7fc664ff6680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:38:09.245 7fc664ff6680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:38:09.245 7fc664ff6680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:09.245 7fc664ff6680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:09.245 7fc664ff6900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:38:09.246 7fc664ff6900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:38:09.247 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:09.248 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:09.248 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:09.248 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:09.248 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:09.249 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:09.250 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:10.251 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:11.251 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:12.253 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:13.254 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:14.255 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:14.352 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:14.353 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:14.353 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:14.354 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:14.354 7fc662e9da40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:14.355 7fc662e9da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:15.256 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:16.257 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:17.258 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:18.260 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:19.261 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:20.262 7fc662eedf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:20.283 7fc662e9da40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:38:20.283 7fc662e9da40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:38:20.284 7fc662e9da40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:20.435 7fc662eed7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:38:20.436 7fc667980d80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:38:20.436 7fc667980d80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:38:20.436 7fc662e9d680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:38:20.436 7fc662e9d680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:38:20.449 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:38:20.449 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:38:20.449 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:38:20.449 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:38:20.449 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:38:20.449 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:38:20.449 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:38:20.449 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:38:20.449 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:38:20.449 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:38:20.450 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:38:20.450 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:38:20.450 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:38:20.450 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:38:20.450 7fc667980d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:38:30.276 7fa06a741c00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:38:30.276 7fa06a741c00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:38:30.276 7fa06a741c00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:38:30.279 7fa06a741c00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:38:30.280 7fa065cba2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:38:30.280 7fa065cba2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:38:30.281 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:38:30.281 7fa065cba7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:38:30.282 7fa065cba7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:38:30.282 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:38:30.282 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.282 7fa065cba7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:38:30.282 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.282 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.282 7fa065cba7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353510282220 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:38:30.283 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.283 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.284 7fa065cba7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:30.284 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.284 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.284 7fa065cba7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:30.284 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.284 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.284 7fa065cba7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:30.284 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.284 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.284 7fa065cba7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:30.284 7fa065cba7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.285 7fa065cbae00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:30.285 7fa065cbae00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:38:30.286 7fa065cbac20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:38:30.287 7fa065c6a540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.287 7fa065c6a540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:30.287 7fa065c6a540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.287 7fa065c6a540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.287 7fa065c6a540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:30.287 7fa065c6a540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.345 7fa065cba7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:38:30.353 7fa065cba7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:38:30.354 7fa065c47040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.355 7fa065c47040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:30.355 7fa065c47040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.355 7fa065c47040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.355 7fa065c47040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:30.355 7fa065c47040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.356 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:30.357 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:30.357 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:30.359 7fa065c6a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.359 7fa065c6a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:30.359 7fa065c6a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.359 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:30.359 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:30.360 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:30.361 7fa065cba7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:38:30.362 7fa065c6a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.362 7fa065c6a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:30.362 7fa065c6a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.363 7fa065c47040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.363 7fa065c47040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:30.363 7fa065c47040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.370 7fa065c47c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.371 7fa065c47c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:30.371 7fa065c47c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.405 7fa065cba7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:30.406 7fa065c47ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.407 7fa065c47ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:30.407 7fa065c47ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.415 7fa065cba7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:30.415 7fa065c47ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.415 7fa065c47ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:30.415 7fa065c47ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.416 7fa065cba7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:38:30.416 7fa065cba7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:38:30.417 7fa065cba7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:38:30.417 7fa065c47e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:30.418 7fa0655850e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.418 7fa0655850e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:30.418 7fa0655850e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.418 7fa0655850e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.418 7fa0655850e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:30.418 7fa0655850e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.423 7fa065cba7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:38:30.426 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:30.428 7fa065c47e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:30.429 7fa065585c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.429 7fa065585c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:30.429 7fa065585c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.429 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:30.429 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:30.430 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:30.430 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:30.431 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:30.456 7fa065cba7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:38:30.458 7fa067dc3400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.458 7fa067dc3400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:30.458 7fa067dc3400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:30.564 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:30.564 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:30.565 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:30.565 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:30.565 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:30.565 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:30.810 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:30.811 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:30.811 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:30.812 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:30.812 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:30.812 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:31.286 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:31.442 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:31.443 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:31.443 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:31.443 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:31.443 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:31.444 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:32.288 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:32.465 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:32.465 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:32.465 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:32.466 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:32.466 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:32.468 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:33.291 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:33.977 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:33.978 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:33.978 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:33.979 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:33.979 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:33.979 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:34.282 7fa065585900 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:38:34.283 7fa065585900 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:38:34.283 7fa065585900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:38:34.283 7fa065585900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:34.283 7fa065585900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:34.283 7fa065585900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:34.283 7fa065585900 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:38:34.283 7fa065585900 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:38:34.284 7fa065585900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:38:34.284 7fa065585900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:34.284 7fa065585900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:34.284 7fa067dc3400 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:38:34.285 7fa067dc3400 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:38:34.293 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:35.294 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:36.294 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:37.296 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:38.089 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:38.090 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:38.090 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:38.091 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:38.091 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:38.091 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:38.297 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:39.298 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:40.300 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:41.301 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:42.301 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:43.302 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:44.303 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:44.358 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:44.359 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:44.359 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:44.359 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:44.360 7fa065c6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:44.360 7fa065c6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:45.305 7fa065cbaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:45.365 7fa065c6aa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:38:45.366 7fa065c6aa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:38:45.366 7fa065c6aa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:45.561 7fa065cba7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:38:45.561 7fa06a741c00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:38:45.561 7fa06a741c00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:38:45.561 7fa065c6a860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:38:45.561 7fa065c6a860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:38:45.577 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:38:45.577 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:38:45.577 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:38:45.577 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:38:45.577 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:38:45.577 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:38:45.578 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:38:45.578 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:38:45.578 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:38:45.578 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:38:45.578 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:38:45.578 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:38:45.578 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:38:45.578 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:38:45.578 7fa06a741c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:38:54.638 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:38:54.638 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:38:54.638 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:38:54.641 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:38:54.642 7ff4482922c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:38:54.642 7ff4482922c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:38:54.642 7ff4482927c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:38:54.642 7ff4482927c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:38:54.643 7ff4482927c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:38:54.643 7ff4482927c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:38:54.643 7ff4482927c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.643 7ff4482927c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:38:54.643 7ff4482927c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.643 7ff4482927c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.643 7ff4482927c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353534643281 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:38:54.643 7ff4482927c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.644 7ff4482927c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.644 7ff4482927c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:54.644 7ff4482927c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.644 7ff4482927c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.645 7ff4482927c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:54.645 7ff4482927c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.645 7ff4482927c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.645 7ff4482927c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:54.645 7ff4482927c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.645 7ff4482927c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.645 7ff4482927c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:54.645 7ff4482927c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.645 7ff448292e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:54.646 7ff448292e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:38:54.647 7ff448292c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:38:54.647 7ff448242540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.647 7ff448242540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:54.647 7ff448242540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.647 7ff448242540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.648 7ff448242540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:54.648 7ff448242540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.775 7ff4482927c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:38:54.784 7ff4482927c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:38:54.785 7ff4481df040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.785 7ff4481df040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:54.785 7ff4481df040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.785 7ff4481df040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.786 7ff4481df040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:54.786 7ff4481df040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.788 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:54.789 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:54.789 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:54.791 7ff4482427c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.792 7ff4482427c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:54.792 7ff4482427c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.793 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:54.793 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:54.793 7ff4482927c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:38:54.793 7ff4482427c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.794 7ff4482427c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:54.794 7ff4482427c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.794 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:54.805 7ff4481dfd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.806 7ff4481dfd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:54.806 7ff4481dfd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.807 7ff4481dfd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.808 7ff4481dfd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:54.808 7ff4481dfd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.833 7ff4482927c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:54.834 7ff4481dfc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.834 7ff4481dfc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:54.834 7ff4481dfc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.850 7ff4482927c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:54.850 7ff4481dfc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.851 7ff4481dfc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:54.851 7ff4481dfc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.851 7ff4482927c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:38:54.851 7ff4482927c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:38:54.851 7ff4482927c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:38:54.852 7ff4481dff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:54.853 7ff448116220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.853 7ff448116220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:54.853 7ff448116220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.853 7ff448116220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.853 7ff448116220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:54.853 7ff448116220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.860 7ff4482927c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:38:54.862 7ff4481dff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:54.863 7ff4481169a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.863 7ff4481169a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:54.863 7ff4481169a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.897 7ff4482927c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:38:54.899 7ff449869180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.899 7ff449869180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:54.899 7ff449869180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:54.921 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:54.922 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:54.922 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:54.923 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:54.923 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:54.923 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:55.144 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:55.145 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:55.145 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:55.146 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:55.146 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:55.147 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:55.531 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:55.532 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:55.532 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:55.533 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:55.533 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:55.533 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:55.647 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:56.595 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:56.596 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:56.596 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:56.597 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:56.597 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:56.597 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:56.647 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:57.648 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:58.361 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:58.362 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:58.362 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:58.363 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:58.363 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:58.363 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:58.643 7ff449869680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:38:58.644 7ff449869680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:38:58.644 7ff449869680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:38:58.644 7ff449869680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:58.644 7ff449869680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:58.645 7ff449869680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:58.645 7ff449869680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:38:58.645 7ff449869680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:38:58.645 7ff449869680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:38:58.645 7ff449869680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:58.645 7ff449869680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:58.645 7ff449869900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:38:58.646 7ff449869900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:38:58.650 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:59.651 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:00.652 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:01.654 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:02.656 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:02.775 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:02.776 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:02.777 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:02.777 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:02.777 7ff448242a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:02.778 7ff448242a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:03.657 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:04.658 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:05.659 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:06.661 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:07.662 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:08.663 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:09.664 7ff448292f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:09.682 7ff448242a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:39:09.682 7ff448242a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:39:09.683 7ff448242a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:09.919 7ff4482927c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:39:09.919 7ff44d2e49c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:39:09.920 7ff44d2e49c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:39:09.920 7ff448242680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:39:09.920 7ff448242680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:39:09.953 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:39:09.954 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:39:09.954 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:39:09.954 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:39:09.954 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:39:09.954 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:39:09.954 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:39:09.954 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:39:09.954 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:39:09.954 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:39:09.955 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:39:09.955 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:39:09.955 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:39:09.955 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:39:09.955 7ff44d2e49c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:39:18.992 7efdf797b180 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:39:18.992 7efdf797b180 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:39:18.992 7efdf797b180 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:39:18.996 7efdf797b180 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:39:18.996 7efdf2ea72c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:39:18.996 7efdf2ea72c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:39:18.996 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:39:18.997 7efdf2ea77c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:39:18.997 7efdf2ea77c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:39:18.997 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:39:18.997 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.997 7efdf2ea77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:39:18.997 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.998 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.998 7efdf2ea77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353558997536 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:39:18.998 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.998 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.998 7efdf2ea77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:18.999 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.999 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.999 7efdf2ea77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:18.999 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.999 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.999 7efdf2ea77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:18.999 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.999 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.999 7efdf2ea77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:18.999 7efdf2ea77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.999 7efdf2ea7f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:39:19.000 7efdf2ea7f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:39:19.001 7efdf2ea7e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:39:19.001 7efdf2e577c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.001 7efdf2e577c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:39:19.001 7efdf2e577c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.001 7efdf2e577c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.002 7efdf2e577c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:19.002 7efdf2e577c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.077 7efdf2ea77c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:39:19.087 7efdf2ea77c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:39:19.088 7efdf2e320e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.088 7efdf2e320e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:19.088 7efdf2e320e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.088 7efdf2e320e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.089 7efdf2e320e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:19.089 7efdf2e320e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.090 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:19.090 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:19.090 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:19.093 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:19.093 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:19.098 7efdf2e579a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.098 7efdf2e579a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:19.098 7efdf2e579a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.099 7efdf2ea77c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:39:19.099 7efdf2e579a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.099 7efdf2e579a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:19.099 7efdf2e579a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.100 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:19.126 7efdf2e32d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.128 7efdf2e32d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:19.128 7efdf2e32d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.132 7efdf2e32d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.133 7efdf2e32d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:19.133 7efdf2e32d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.145 7efdf2ea77c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:39:19.146 7efdf2e32c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.146 7efdf2e32c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:19.146 7efdf2e32c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.149 7efdf2ea77c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:39:19.149 7efdf2e32c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.150 7efdf2e32c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:19.150 7efdf2e32c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.150 7efdf2ea77c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:39:19.150 7efdf2ea77c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:39:19.150 7efdf2ea77c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:39:19.151 7efdf2e32f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:39:19.151 7efdf2776220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.151 7efdf2776220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:19.151 7efdf2776220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.151 7efdf2776220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.152 7efdf2776220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:19.152 7efdf2776220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.156 7efdf2ea77c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:39:19.158 7efdf2e32f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:39:19.159 7efdf27769a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.159 7efdf27769a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:19.159 7efdf27769a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.192 7efdf2ea77c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:39:19.196 7efdf4fb1180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.196 7efdf4fb1180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:39:19.196 7efdf4fb1180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:19.233 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:19.234 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:19.234 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:19.235 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:19.235 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:19.235 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:19.489 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:19.490 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:19.490 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:19.491 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:19.491 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:19.491 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:20.001 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:20.015 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:20.016 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:20.016 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:20.016 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:20.016 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:20.017 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:20.774 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:20.776 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:20.776 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:20.776 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:20.776 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:20.777 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:21.002 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:22.003 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:22.690 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:22.691 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:22.691 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:22.692 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:22.692 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:22.693 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:22.997 7efdf4fb1680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:39:22.998 7efdf4fb1680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:39:22.999 7efdf4fb1680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:39:22.999 7efdf4fb1680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:22.999 7efdf4fb1680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:39:22.999 7efdf4fb1680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:22.999 7efdf4fb1680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:39:22.999 7efdf4fb1680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:39:22.999 7efdf4fb1680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:39:23.000 7efdf4fb1680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:23.000 7efdf4fb1680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:23.000 7efdf4fb1900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:39:23.001 7efdf4fb1900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:39:23.004 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:24.006 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:25.007 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:25.528 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:25.529 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:25.529 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:25.530 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:25.530 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:25.531 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:26.008 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:27.009 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:28.010 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:29.011 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:30.012 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:30.277 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:30.278 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:30.278 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:30.279 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:30.279 7efdf2e57c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:30.279 7efdf2e57c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:31.013 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:32.014 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:33.016 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:34.017 7efdf2e572c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:34.097 7efdf2e57c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:39:34.097 7efdf2e57c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:39:34.098 7efdf2e57c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:34.257 7efdf2ea77c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:39:34.258 7efdf797b180 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:39:34.258 7efdf797b180 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:39:34.258 7efdf2e57860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:39:34.258 7efdf2e57860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:39:34.272 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:39:34.273 7efdf797b180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:39:43.184 7f57dc3e2980 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:39:43.184 7f57dc3e2980 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:39:43.184 7f57dc3e2980 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:39:43.190 7f57dc3e2980 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:39:43.191 7f57d76272c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:39:43.191 7f57d76272c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:39:43.191 7f57d76277c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:39:43.192 7f57d76277c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:39:43.192 7f57d76277c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:39:43.192 7f57d76277c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:39:43.192 7f57d76277c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.192 7f57d76277c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:39:43.193 7f57d76277c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.193 7f57d76277c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.193 7f57d76277c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353583192467 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:39:43.193 7f57d76277c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.193 7f57d76277c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.194 7f57d76277c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:43.194 7f57d76277c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.194 7f57d76277c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.194 7f57d76277c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:43.194 7f57d76277c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.194 7f57d76277c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.194 7f57d76277c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:43.194 7f57d76277c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.194 7f57d76277c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.194 7f57d76277c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:43.194 7f57d76277c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.195 7f57d7627e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:39:43.195 7f57d7627e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:39:43.196 7f57d7627c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:39:43.196 7f57d75d6540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.196 7f57d75d6540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:39:43.197 7f57d75d6540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.197 7f57d75d6540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.197 7f57d75d6540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:43.197 7f57d75d6540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.257 7f57d76277c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:39:43.266 7f57d76277c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:39:43.267 7f57d9737040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.267 7f57d9737040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:43.267 7f57d9737040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.267 7f57d9737040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.267 7f57d9737040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:43.267 7f57d9737040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.268 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:43.269 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:43.269 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:43.271 7f57d75d67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.271 7f57d75d67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:43.271 7f57d75d67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.271 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:43.271 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:43.272 7f57d76277c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:39:43.273 7f57d75d67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.273 7f57d75d67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:43.273 7f57d75d67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.273 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:43.281 7f57d9737c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.282 7f57d9737c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:43.282 7f57d9737c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.284 7f57d9737c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.284 7f57d9737c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:43.284 7f57d9737c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.310 7f57d76277c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:39:43.311 7f57d9737ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.311 7f57d9737ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:43.311 7f57d9737ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.317 7f57d76277c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:39:43.317 7f57d9737ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.317 7f57d9737ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:43.317 7f57d9737ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.318 7f57d76277c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:39:43.318 7f57d76277c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:39:43.319 7f57d76277c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:39:43.319 7f57d9737e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:39:43.319 7f57d8bff0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.320 7f57d8bff0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:43.320 7f57d8bff0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.320 7f57d8bff0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.320 7f57d8bff0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:43.320 7f57d8bff0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.323 7f57d76277c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:39:43.327 7f57d9737e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:39:43.328 7f57d8bff360 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.328 7f57d8bff360 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:43.329 7f57d8bff360 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.350 7f57d76277c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:39:43.351 7f57d8bd7400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.351 7f57d8bd7400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:39:43.351 7f57d8bd7400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:43.352 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:43.353 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:43.353 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:43.354 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:43.354 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:43.354 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:43.472 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:43.473 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:43.474 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:43.474 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:43.474 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:43.475 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:43.730 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:43.731 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:43.731 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:43.731 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:43.731 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:43.732 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:44.196 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:44.224 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:44.225 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:44.225 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:44.225 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:44.225 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:44.226 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:44.829 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:44.830 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:44.830 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:44.830 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:44.830 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:44.831 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:45.197 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:45.720 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:45.720 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:45.721 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:45.721 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:45.721 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:45.722 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:46.198 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:47.122 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:47.123 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:47.123 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:47.123 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:47.123 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:47.124 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:47.192 7f57d8bd7400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:39:47.194 7f57d8bd7400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:39:47.194 7f57d8bd7400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:39:47.194 7f57d8bd7400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:47.195 7f57d8bd7400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:39:47.195 7f57d8bd7400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:47.195 7f57d8bd7400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:39:47.195 7f57d8bd7400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:39:47.196 7f57d8bd7400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:39:47.196 7f57d8bd7400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:47.196 7f57d8bd7400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:47.196 7f57d8bd7900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:39:47.197 7f57d8bd7900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:39:47.199 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:48.200 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:49.028 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:49.029 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:49.029 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:49.029 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:49.030 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:49.030 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:49.202 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:50.203 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:51.204 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:52.205 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:53.207 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:54.207 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:54.232 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:54.232 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:54.233 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:54.233 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:54.233 7f57d75d6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:54.234 7f57d75d6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:55.208 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:56.209 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:57.210 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:58.211 7f57d7627f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:58.252 7f57d75d6a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:39:58.252 7f57d75d6a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:39:58.253 7f57d75d6a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:58.386 7f57d76277c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:39:58.387 7f57dc3e2980 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:39:58.387 7f57dc3e2980 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:39:58.387 7f57d75d6680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:39:58.387 7f57d75d6680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:39:58.400 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:39:58.400 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:39:58.400 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:39:58.401 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:39:58.401 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:39:58.401 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:39:58.401 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:39:58.401 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:39:58.401 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:39:58.401 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:39:58.401 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:39:58.401 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:39:58.401 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:39:58.401 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:39:58.401 7f57dc3e2980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:40:07.235 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:40:07.235 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:40:07.235 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:40:07.239 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:40:07.240 7f87030a22c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:40:07.240 7f87030a22c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:40:07.240 7f87030a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:40:07.240 7f87030a27c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:40:07.241 7f87030a27c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:40:07.241 7f87030a27c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:40:07.241 7f87030a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.241 7f87030a27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:40:07.241 7f87030a27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.241 7f87030a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.242 7f87030a27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353607241104 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:40:07.242 7f87030a27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.243 7f87030a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.243 7f87030a27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:07.243 7f87030a27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.243 7f87030a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.243 7f87030a27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:07.243 7f87030a27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.243 7f87030a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.243 7f87030a27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:07.243 7f87030a27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.244 7f87030a27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.244 7f87030a27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:07.244 7f87030a27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.244 7f87030a2e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:07.244 7f87030a2e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:40:07.245 7f87030a2c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:40:07.246 7f8703052540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.246 7f8703052540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:07.246 7f8703052540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.246 7f8703052540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.246 7f8703052540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:07.246 7f8703052540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.299 7f87030a27c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:40:07.307 7f87030a27c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:40:07.308 7f8702faf0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.308 7f8702faf0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:07.308 7f8702faf0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.308 7f8702faf0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.308 7f8702faf0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:07.308 7f8702faf0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.310 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:07.311 7f87030527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.312 7f87030527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:07.312 7f87030527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.313 7f87030a27c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:40:07.314 7f87030527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.314 7f87030527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:07.314 7f87030527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.314 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.315 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:07.315 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.316 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:07.322 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.324 7f8702fafd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.325 7f8702fafd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:07.325 7f8702fafd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.327 7f8702fafd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.327 7f8702fafd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:07.327 7f8702fafd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.350 7f87030a27c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:07.350 7f8702fafc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.351 7f8702fafc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:07.351 7f8702fafc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.354 7f87030a27c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:07.355 7f8702fafc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.355 7f8702fafc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:07.355 7f8702fafc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.355 7f87030a27c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:40:07.355 7f87030a27c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:40:07.355 7f87030a27c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:40:07.356 7f8702faff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:07.356 7f8702f28220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.357 7f8702f28220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:07.357 7f8702f28220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.357 7f8702f28220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.357 7f8702f28220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:07.357 7f8702f28220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.360 7f87030a27c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:40:07.362 7f8702faff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:07.362 7f8702f289a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.362 7f8702f289a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:07.362 7f8702f289a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.384 7f87030a27c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:40:07.386 7f8704737400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.387 7f8704737400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:07.387 7f8704737400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:07.410 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:07.411 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.411 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:07.411 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.411 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:07.412 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.538 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:07.539 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.539 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:07.539 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.539 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:07.540 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.747 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:07.748 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.748 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:07.749 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.749 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:07.749 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:08.245 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:08.289 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:08.290 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:08.290 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:08.290 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:08.290 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:08.291 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:09.246 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:09.270 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:09.271 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:09.271 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:09.272 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:09.272 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:09.272 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:10.247 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:11.241 7f8704737400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:40:11.242 7f8704737400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:40:11.242 7f8704737400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:40:11.242 7f8704737400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:11.243 7f8704737400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:11.243 7f8704737400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:11.243 7f8704737400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:40:11.243 7f8704737400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:40:11.243 7f8704737400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:40:11.244 7f8704737400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:11.244 7f8704737400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:11.244 7f8704737900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:40:11.245 7f8704737900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:40:11.249 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:11.487 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:11.488 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:11.488 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:11.489 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:11.489 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:11.489 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:12.250 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:13.252 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:14.253 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:15.254 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:16.255 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:17.256 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:17.621 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:17.622 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:17.622 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:17.623 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:17.623 7f8703052c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:17.623 7f8703052c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:18.257 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:19.259 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:20.260 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:21.262 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:22.262 7f87030a2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:22.346 7f8703052c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:40:22.347 7f8703052c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:40:22.347 7f8703052c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:22.397 7f87030a27c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:40:22.397 7f8707b2f1c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:40:22.397 7f8707b2f1c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:40:22.398 7f8703052680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:40:22.398 7f8703052680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:40:22.411 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:40:22.411 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:40:22.411 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:40:22.411 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:40:22.411 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:40:22.411 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:40:22.411 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:40:22.411 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:40:22.412 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:40:22.412 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:40:22.412 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:40:22.412 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:40:22.412 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:40:22.412 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:40:22.412 7f8707b2f1c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:40:31.594 7f101c3de500 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:40:31.594 7f101c3de500 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:40:31.594 7f101c3de500 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:40:31.598 7f101c3de500 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:40:31.598 7f10176402c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:40:31.599 7f10176402c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:40:31.599 7f10176407c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:40:31.599 7f10176407c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:40:31.600 7f10176407c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:40:31.600 7f10176407c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:40:31.600 7f10176407c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.600 7f10176407c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:40:31.600 7f10176407c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.600 7f10176407c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.600 7f10176407c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353631599964 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:40:31.600 7f10176407c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.601 7f10176407c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.601 7f10176407c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:31.601 7f10176407c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.602 7f10176407c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.602 7f10176407c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:31.602 7f10176407c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.602 7f10176407c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.602 7f10176407c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:31.602 7f10176407c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.602 7f10176407c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.602 7f10176407c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:31.602 7f10176407c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.603 7f1017640e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:31.603 7f1017640e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:40:31.604 7f1017640c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:40:31.605 7f10175f0540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.605 7f10175f0540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:31.605 7f10175f0540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.605 7f10175f0540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.605 7f10175f0540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:31.605 7f10175f0540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.670 7f10176407c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:40:31.679 7f10176407c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:40:31.680 7f101754e040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.680 7f101754e040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:31.680 7f101754e040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.680 7f101754e040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.680 7f101754e040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:31.681 7f101754e040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.683 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:31.684 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:31.684 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:31.684 7f10176407c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:40:31.685 7f10175f07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.685 7f10175f07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:31.685 7f10175f07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.686 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:31.686 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:31.693 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:31.694 7f101754ec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.694 7f101754ec20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:31.694 7f101754ec20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.696 7f101754ec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.696 7f101754ec20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:31.696 7f101754ec20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.698 7f101754ec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.698 7f101754ec20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:31.698 7f101754ec20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.724 7f10176407c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:31.725 7f101754eae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.725 7f101754eae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:31.725 7f101754eae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.728 7f10176407c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:31.729 7f101754eae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.729 7f101754eae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:31.729 7f101754eae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.729 7f10176407c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:40:31.730 7f10176407c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:40:31.730 7f10176407c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:40:31.730 7f101754ee00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:31.731 7f10174c70e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.731 7f10174c70e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:31.731 7f10174c70e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.731 7f10174c70e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.731 7f10174c70e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:31.731 7f10174c70e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.735 7f10176407c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:40:31.737 7f101754ee00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:31.737 7f10174c77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.737 7f10174c77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:31.738 7f10174c77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.764 7f10176407c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:40:31.766 7f10174c7ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.766 7f10174c7ea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:31.766 7f10174c7ea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:31.831 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:31.832 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:31.832 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:31.833 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:31.833 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:31.833 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:32.118 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:32.119 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:32.119 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:32.120 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:32.120 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:32.120 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:32.604 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:32.816 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:32.817 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:32.817 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:32.818 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:32.818 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:32.818 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:33.605 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:33.854 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:33.855 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:33.855 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:33.855 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:33.856 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:33.856 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:34.607 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:35.600 7f10174c7ea0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:40:35.601 7f10174c7ea0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:40:35.601 7f10174c7ea0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:40:35.602 7f10174c7ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:35.602 7f10174c7ea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:35.602 7f10174c7ea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:35.602 7f10174c7ea0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:40:35.602 7f10174c7ea0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:40:35.602 7f10174c7ea0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:40:35.602 7f10174c7ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:35.602 7f10174c7ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:35.603 7f1018cc4180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:40:35.603 7f1018cc4180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:40:35.608 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:36.049 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:36.053 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:36.053 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:36.053 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:36.053 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:36.054 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:36.609 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:37.610 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:38.611 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:39.612 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:40.614 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:40.956 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:40.956 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:40.956 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:40.957 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:40.957 7f10175f0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:40.958 7f10175f0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:41.615 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:42.616 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:43.618 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:44.619 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:45.628 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:46.629 7f1017640f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:46.685 7f10175f0a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:40:46.685 7f10175f0a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:40:46.686 7f10175f0a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:46.807 7f10176407c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:40:46.808 7f101c3de500 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:40:46.808 7f101c3de500 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:40:46.808 7f10175f0860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:40:46.808 7f10175f0860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:40:46.821 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:40:46.822 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:40:46.822 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:40:46.822 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:40:46.822 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:40:46.822 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:40:46.822 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:40:46.822 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:40:46.822 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:40:46.822 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:40:46.822 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:40:46.822 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:40:46.822 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:40:46.822 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:40:46.823 7f101c3de500 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:40:56.175 7fe49fbd9800 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:40:56.175 7fe49fbd9800 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:40:56.175 7fe49fbd9800 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:40:56.178 7fe49fbd9800 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:40:56.178 7fe49b1532c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:40:56.179 7fe49b1532c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:40:56.179 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:40:56.179 7fe49b1537c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:40:56.180 7fe49b1537c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:40:56.180 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:40:56.180 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.180 7fe49b1537c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:40:56.181 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.181 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.181 7fe49b1537c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353656179967 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:40:56.181 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.182 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.182 7fe49b1537c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:56.182 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.182 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.182 7fe49b1537c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:56.183 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.183 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.183 7fe49b1537c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:56.183 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.183 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.183 7fe49b1537c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:56.183 7fe49b1537c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.183 7fe49b153f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:56.184 7fe49b153f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:40:56.185 7fe49b153e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:40:56.186 7fe49b1037c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.186 7fe49b1037c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:56.187 7fe49b1037c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.187 7fe49b1037c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.187 7fe49b1037c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:56.187 7fe49b1037c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.253 7fe49b1537c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:40:56.262 7fe49b1537c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:40:56.263 7fe49b0e40e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.263 7fe49b0e40e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:56.263 7fe49b0e40e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.263 7fe49b0e40e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.263 7fe49b0e40e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:56.264 7fe49b0e40e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.267 7fe49b1039a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.268 7fe49b1039a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:56.268 7fe49b1039a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.269 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:56.271 7fe49b1537c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:40:56.271 7fe49b1039a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.271 7fe49b1039a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:56.272 7fe49b1039a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.272 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:56.272 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:56.287 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:56.287 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:56.288 7fe49b0e4d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.289 7fe49b0e4d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:56.289 7fe49b0e4d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.289 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:56.291 7fe49b0e4d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.291 7fe49b0e4d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:56.291 7fe49b0e4d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.312 7fe49b1537c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:56.313 7fe49b0e4c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.313 7fe49b0e4c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:56.313 7fe49b0e4c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.316 7fe49b1537c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:56.320 7fe49b0e4c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.321 7fe49b0e4c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:56.321 7fe49b0e4c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.321 7fe49b1537c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:40:56.321 7fe49b1537c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:40:56.321 7fe49b1537c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:40:56.322 7fe49b0e4f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:56.325 7fe49afd8220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.325 7fe49afd8220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:56.325 7fe49afd8220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.325 7fe49afd8220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.325 7fe49afd8220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:56.325 7fe49afd8220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.330 7fe49b1537c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:40:56.332 7fe49b0e4f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:56.333 7fe49afd85e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.334 7fe49afd85e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:56.334 7fe49afd85e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.369 7fe49b1537c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:40:56.371 7fe49afd8ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.371 7fe49afd8ea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:56.371 7fe49afd8ea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:56.372 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:56.372 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:56.373 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:56.373 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:56.374 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:56.374 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:56.504 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:56.525 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:56.525 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:56.526 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:56.526 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:56.526 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:56.796 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:56.797 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:56.797 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:56.798 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:56.798 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:56.799 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:57.185 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:57.437 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:57.437 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:57.438 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:57.438 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:57.438 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:57.439 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:58.186 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:58.842 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:58.843 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:58.843 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:58.843 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:58.844 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:58.844 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:59.187 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:00.179 7fe49afd8ea0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:41:00.180 7fe49afd8ea0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:41:00.181 7fe49afd8ea0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:41:00.181 7fe49afd8ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:00.181 7fe49afd8ea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:00.181 7fe49afd8ea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:00.181 7fe49afd8ea0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:41:00.181 7fe49afd8ea0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:41:00.181 7fe49afd8ea0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:41:00.181 7fe49afd8ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:00.182 7fe49afd8ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:00.182 7fe49d253180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:41:00.183 7fe49d253180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:41:00.188 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:01.189 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:01.782 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:01.783 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:01.783 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:01.784 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:01.784 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:01.788 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:02.189 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:03.191 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:04.192 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:05.193 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:06.194 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:07.195 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:08.196 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:09.178 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:09.179 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:09.179 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:09.180 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:09.180 7fe49b103c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:09.180 7fe49b103c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:09.198 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:10.199 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:11.190 7fe49b103c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:41:11.191 7fe49b103c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:41:11.191 7fe49b103c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:11.200 7fe49b1032c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:11.421 7fe49b1537c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:41:11.422 7fe49fbd9800 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:41:11.422 7fe49fbd9800 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:41:11.422 7fe49b103b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:41:11.422 7fe49b103b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:41:11.437 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:41:11.437 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:41:11.437 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:41:11.437 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:41:11.437 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:41:11.437 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:41:11.437 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:41:11.437 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:41:11.437 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:41:11.437 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:41:11.437 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:41:11.438 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:41:11.438 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:41:11.438 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:41:11.438 7fe49fbd9800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:41:20.676 7fd565dacc00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:41:20.676 7fd565dacc00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:41:20.676 7fd565dacc00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:41:20.679 7fd565dacc00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:41:20.680 7fd56131b2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:41:20.680 7fd56131b2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:41:20.680 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:41:20.680 7fd56131b7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:41:20.680 7fd56131b7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:41:20.681 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:41:20.681 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.681 7fd56131b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:41:20.681 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.681 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.681 7fd56131b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353680680903 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:41:20.681 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.682 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.682 7fd56131b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:20.682 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.682 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.682 7fd56131b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:20.682 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.682 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.682 7fd56131b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:20.682 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.682 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.682 7fd56131b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:20.682 7fd56131b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.683 7fd56131be00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:20.683 7fd56131be00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:41:20.684 7fd56131bc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:41:20.685 7fd5612cb540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.685 7fd5612cb540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:20.685 7fd5612cb540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.685 7fd5612cb540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.685 7fd5612cb540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:20.685 7fd5612cb540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.741 7fd56131b7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:41:20.748 7fd56131b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:41:20.749 7fd561198040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.749 7fd561198040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:20.749 7fd561198040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.749 7fd561198040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.749 7fd561198040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:20.749 7fd561198040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.752 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:20.753 7fd5612cb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.753 7fd5612cb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:20.753 7fd5612cb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.754 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:20.754 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:20.755 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:20.755 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:20.755 7fd56131b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:41:20.755 7fd5612cb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.755 7fd5612cb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:20.756 7fd5612cb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.761 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:20.763 7fd561198c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.763 7fd561198c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:20.763 7fd561198c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.765 7fd561198c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.765 7fd561198c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:20.765 7fd561198c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.795 7fd56131b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:20.795 7fd561198ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.796 7fd561198ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:20.796 7fd561198ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.804 7fd56131b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:20.804 7fd561198ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.805 7fd561198ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:20.805 7fd561198ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.805 7fd56131b7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:41:20.805 7fd56131b7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:41:20.805 7fd56131b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:41:20.805 7fd561198e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:20.806 7fd5628ee0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.806 7fd5628ee0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:20.806 7fd5628ee0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.807 7fd5628ee0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.807 7fd5628ee0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:20.807 7fd5628ee0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.810 7fd56131b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:41:20.812 7fd561198e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:20.813 7fd5628ee860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.813 7fd5628ee860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:20.813 7fd5628ee860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.836 7fd56131b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:41:20.838 7fd5628cd180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.838 7fd5628cd180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:20.838 7fd5628cd180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:20.876 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:20.876 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:20.876 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:20.877 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:20.877 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:20.882 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:21.082 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:21.083 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:21.083 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:21.083 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:21.083 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:21.084 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:21.606 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:21.607 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:21.607 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:21.607 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:21.608 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:21.608 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:21.684 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:22.447 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:22.448 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:22.448 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:22.449 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:22.449 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:22.449 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:22.708 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:23.710 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:23.940 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:23.941 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:23.941 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:23.942 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:23.942 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:23.942 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:24.681 7fd5628cd680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:41:24.683 7fd5628cd680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:41:24.683 7fd5628cd680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:41:24.683 7fd5628cd680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:24.683 7fd5628cd680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:24.683 7fd5628cd680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:24.683 7fd5628cd680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:41:24.683 7fd5628cd680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:41:24.683 7fd5628cd680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:41:24.684 7fd5628cd680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:24.684 7fd5628cd680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:24.684 7fd5628cd900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:41:24.685 7fd5628cd900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:41:24.711 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:25.713 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:26.714 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:27.715 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:27.936 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:27.937 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:27.937 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:27.937 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:27.938 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:27.938 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:28.716 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:29.717 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:30.718 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:31.720 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:32.721 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:33.722 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:33.729 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:33.730 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:33.730 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:33.731 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:33.731 7fd5612cba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:33.731 7fd5612cba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:34.723 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:35.725 7fd56131bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:35.741 7fd5612cba40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:41:35.741 7fd5612cba40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:41:35.742 7fd5612cba40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:35.866 7fd56131b7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:41:35.866 7fd565dacc00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:41:35.867 7fd565dacc00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:41:35.867 7fd5612cb860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:41:35.867 7fd5612cb860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:41:35.886 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:41:35.886 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:41:35.886 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:41:35.886 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:41:35.886 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:41:35.887 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:41:35.887 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:41:35.887 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:41:35.887 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:41:35.887 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:41:35.887 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:41:35.887 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:41:35.887 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:41:35.887 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:41:35.887 7fd565dacc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:41:47.346 7fd24494c9c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:41:47.347 7fd24494c9c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:41:47.347 7fd24494c9c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:41:47.355 7fd24494c9c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:41:47.356 7fd23f8fb2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:41:47.356 7fd23f8fb2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:41:47.356 7fd23f8fb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:41:47.357 7fd23f8fb7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:41:47.357 7fd23f8fb7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:41:47.357 7fd23f8fb7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:41:47.357 7fd23f8fb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.357 7fd23f8fb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:41:47.358 7fd23f8fb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.358 7fd23f8fb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.358 7fd23f8fb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353707357474 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:41:47.358 7fd23f8fb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.362 7fd23f8fb900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.364 7fd23f8fb900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:47.364 7fd23f8fb900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.364 7fd23f8fb900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.365 7fd23f8fb900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:47.365 7fd23f8fb900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.365 7fd23f8fb900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.365 7fd23f8fb900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:47.365 7fd23f8fb900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.365 7fd23f8fb900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.365 7fd23f8fb900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:47.365 7fd23f8fb900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.367 7fd23f8fbf40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:47.367 7fd23f8fbf40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:41:47.369 7fd23f8fbe00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:41:47.369 7fd23f8ab7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.369 7fd23f8ab7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:47.369 7fd23f8ab7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.369 7fd23f8ab7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.369 7fd23f8ab7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:47.369 7fd23f8ab7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.504 7fd23f8fb900 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:41:47.523 7fd23f8fb900 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:41:47.525 7fd23f77c0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.526 7fd23f77c0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:47.526 7fd23f77c0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.527 7fd23f77c0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.527 7fd23f77c0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:47.527 7fd23f77c0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.532 7fd23f77c0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.532 7fd23f77c0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:47.532 7fd23f77c0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.533 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:47.534 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:47.535 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:47.538 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:47.538 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:47.540 7fd23f8fb900 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:41:47.541 7fd23f8ab9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.541 7fd23f8ab9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:47.541 7fd23f8ab9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.541 7fd23f8ab9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.541 7fd23f8ab9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:47.541 7fd23f8ab9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.541 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:47.558 7fd23f77cd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.559 7fd23f77cd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:47.559 7fd23f77cd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.585 7fd23f8fb900 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:47.586 7fd23f77cc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.586 7fd23f77cc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:47.586 7fd23f77cc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.590 7fd23f8fb900 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:47.591 7fd23f77cc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.591 7fd23f77cc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:47.591 7fd23f77cc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.591 7fd23f8fb900 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:41:47.591 7fd23f8fb900 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:41:47.591 7fd23f8fb900 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:41:47.592 7fd23f77cf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:47.592 7fd240ed1220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.592 7fd240ed1220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:47.593 7fd240ed1220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.593 7fd240ed1220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.593 7fd240ed1220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:47.593 7fd240ed1220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.596 7fd23f8fb900 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:41:47.601 7fd23f77cf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:47.602 7fd240ed19a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.602 7fd240ed19a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:47.602 7fd240ed19a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.638 7fd23f8fb900 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:41:47.654 7fd240eab400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.654 7fd240eab400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:47.654 7fd240eab400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:47.655 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:47.669 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:47.670 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:47.677 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:47.677 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:47.682 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:47.847 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:47.848 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:47.848 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:47.849 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:47.849 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:47.849 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:48.272 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:48.272 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:48.272 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:48.273 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:48.273 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:48.274 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:48.368 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:49.327 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:49.328 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:49.328 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:49.328 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:49.329 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:49.329 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:49.370 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:50.372 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:51.358 7fd240eab400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:41:51.370 7fd240eab400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:41:51.370 7fd240eab400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:41:51.370 7fd240eab400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:51.371 7fd240eab400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:51.371 7fd240eab400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:51.371 7fd240eab400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:41:51.371 7fd240eab400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:41:51.371 7fd240eab400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:41:51.371 7fd240eab400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:51.371 7fd240eab400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:51.372 7fd240eab900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:41:51.373 7fd240eab900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:41:51.373 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:52.174 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:52.174 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:52.175 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:52.175 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:52.175 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:52.176 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:52.374 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:53.376 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:54.377 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:55.378 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:56.380 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:57.381 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:58.038 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:58.039 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:58.039 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:58.040 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:58.040 7fd23f8abc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:58.040 7fd23f8abc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:58.382 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:59.384 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:00.385 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:01.386 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:02.387 7fd23f8ab2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:02.460 7fd23f8abc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:42:02.460 7fd23f8abc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:42:02.461 7fd23f8abc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:02.690 7fd23f8fb900 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:42:02.690 7fd24494c9c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:42:02.690 7fd24494c9c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:42:02.690 7fd23f8ab860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:42:02.690 7fd23f8ab860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:42:02.712 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:42:02.712 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:42:02.712 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:42:02.712 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:42:02.712 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:42:02.712 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:42:02.712 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:42:02.713 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:42:02.713 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:42:02.713 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:42:02.713 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:42:02.714 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:42:02.714 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:42:02.714 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:42:02.714 7fd24494c9c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:42:13.386 7fcef38bed00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:42:13.386 7fcef38bed00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:42:13.386 7fcef38bed00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:42:13.391 7fcef38bed00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:42:13.391 7fceee8752c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:42:13.391 7fceee8752c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:42:13.392 7fceee8757c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:42:13.392 7fceee8757c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:42:13.393 7fceee8757c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:42:13.393 7fceee8757c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:42:13.393 7fceee8757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.393 7fceee8757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:42:13.393 7fceee8757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.393 7fceee8757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.393 7fceee8757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353733392971 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:42:13.393 7fceee8757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.394 7fceee8757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.394 7fceee8757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:13.394 7fceee8757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.394 7fceee8757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.395 7fceee8757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:13.395 7fceee8757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.395 7fceee8757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.395 7fceee8757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:13.395 7fceee8757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.395 7fceee8757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.395 7fceee8757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:13.395 7fceee8757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.395 7fceee875e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:42:13.395 7fceee875e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:42:13.397 7fceee875c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:42:13.397 7fceee825540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.397 7fceee825540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:42:13.397 7fceee825540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.397 7fceee825540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.397 7fceee825540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:13.398 7fceee825540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.465 7fceee8757c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:42:13.475 7fceee8757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:42:13.476 7fcef0984040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.476 7fcef0984040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:13.476 7fcef0984040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.476 7fcef0984040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.476 7fcef0984040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:13.476 7fcef0984040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.478 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:13.480 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:13.481 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:13.481 7fceee8257c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.481 7fceee8257c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:13.481 7fceee8257c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.482 7fceee8757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:42:13.483 7fceee8257c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.483 7fceee8257c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:13.483 7fceee8257c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.484 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:13.484 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:13.485 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:13.494 7fcef0984c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.495 7fcef0984c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:13.495 7fcef0984c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.496 7fcef0984c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.496 7fcef0984c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:13.496 7fcef0984c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.517 7fceee8757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:42:13.518 7fcef0984ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.519 7fcef0984ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:13.519 7fcef0984ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.523 7fceee8757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:42:13.523 7fcef0984ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.524 7fcef0984ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:13.524 7fcef0984ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.524 7fceee8757c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:42:13.524 7fceee8757c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:42:13.525 7fceee8757c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:42:13.525 7fcef0984e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:42:13.526 7fceee7fb0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.527 7fceee7fb0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:13.527 7fceee7fb0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.527 7fceee7fb0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.527 7fceee7fb0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:13.527 7fceee7fb0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.531 7fceee8757c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:42:13.534 7fcef0984e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:42:13.535 7fceee7fb860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.535 7fceee7fb860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:13.535 7fceee7fb860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.564 7fceee8757c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:42:13.566 7fceefe26180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.566 7fceefe26180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:42:13.566 7fceefe26180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:13.619 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:13.620 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:13.620 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:13.621 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:13.621 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:13.622 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:13.954 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:13.955 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:13.955 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:13.956 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:13.956 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:13.956 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:14.396 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:14.715 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:14.716 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:14.716 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:14.716 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:14.716 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:14.717 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:15.397 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:16.399 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:16.572 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:16.573 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:16.573 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:16.573 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:16.573 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:16.574 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:17.393 7fceefe26680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:42:17.394 7fceefe26680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:42:17.395 7fceefe26680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:42:17.395 7fceefe26680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:17.395 7fceefe26680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:42:17.395 7fceefe26680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:17.395 7fceefe26680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:42:17.395 7fceefe26680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:42:17.395 7fceefe26680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:42:17.395 7fceefe26680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:17.396 7fceefe26680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:17.396 7fceefe26900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:42:17.397 7fceefe26900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:42:17.401 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:18.401 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:19.403 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:20.207 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:20.207 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:20.207 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:20.208 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:20.208 7fceee825a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:20.208 7fceee825a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:20.404 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:21.405 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:22.407 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:23.408 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:24.409 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:25.410 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:26.411 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:27.412 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:28.414 7fceee875f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:28.451 7fceee825a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:42:28.451 7fceee825a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:42:28.451 7fceee825a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:28.647 7fceee8757c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:42:28.647 7fcef38bed00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:42:28.647 7fcef38bed00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:42:28.647 7fceee825860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:42:28.648 7fceee825860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:42:28.663 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:42:28.663 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:42:28.663 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:42:28.663 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:42:28.663 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:42:28.663 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:42:28.663 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:42:28.663 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:42:28.663 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:42:28.664 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:42:28.664 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:42:28.664 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:42:28.664 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:42:28.664 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:42:28.664 7fcef38bed00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:42:38.311 7f950d051c00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:42:38.311 7f950d051c00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:42:38.311 7f950d051c00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:42:38.315 7f950d051c00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:42:38.315 7f95085c72c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:42:38.316 7f95085c72c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:42:38.316 7f95085c77c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:42:38.316 7f95085c77c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:42:38.317 7f95085c77c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:42:38.317 7f95085c77c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:42:38.317 7f95085c77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.317 7f95085c77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:42:38.317 7f95085c77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.317 7f95085c77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.317 7f95085c77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353758317165 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:42:38.317 7f95085c77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.318 7f95085c77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.318 7f95085c77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:38.318 7f95085c77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.319 7f95085c77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.319 7f95085c77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:38.319 7f95085c77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.319 7f95085c77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.319 7f95085c77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:38.319 7f95085c77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.319 7f95085c77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.319 7f95085c77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:38.319 7f95085c77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.320 7f95085c7e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:42:38.320 7f95085c7e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:42:38.321 7f95085c7c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:42:38.322 7f9508577540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.322 7f9508577540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:42:38.322 7f9508577540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.322 7f9508577540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.322 7f9508577540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:38.322 7f9508577540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.385 7f95085c77c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:42:38.394 7f95085c77c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:42:38.395 7f9508559040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.395 7f9508559040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:38.395 7f9508559040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.396 7f9508559040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.396 7f9508559040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:38.396 7f9508559040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.397 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:38.399 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:38.400 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:38.400 7f95085c77c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:42:38.400 7f95085777c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.401 7f95085777c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:38.401 7f95085777c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.402 7f95085777c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.402 7f95085777c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:38.402 7f95085777c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.403 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:38.403 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:38.414 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:38.416 7f9508559c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.416 7f9508559c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:38.416 7f9508559c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.418 7f9508559c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.418 7f9508559c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:38.418 7f9508559c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.469 7f95085c77c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:42:38.470 7f9508559ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.470 7f9508559ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:38.470 7f9508559ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.473 7f95085c77c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:42:38.474 7f9508559ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.474 7f9508559ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:38.474 7f9508559ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.474 7f95085c77c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:42:38.474 7f95085c77c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:42:38.475 7f95085c77c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:42:38.475 7f9508559e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:42:38.475 7f950a6ce0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.476 7f950a6ce0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:38.476 7f950a6ce0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.476 7f950a6ce0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.476 7f950a6ce0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:38.476 7f950a6ce0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.480 7f95085c77c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:42:38.481 7f9508559e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:42:38.482 7f950a6ce860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.482 7f950a6ce860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:38.482 7f950a6ce860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.507 7f95085c77c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:42:38.509 7f9507e93400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.510 7f9507e93400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:42:38.510 7f9507e93400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:38.545 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:38.545 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:38.546 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:38.546 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:38.546 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:38.547 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:38.891 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:38.891 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:38.891 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:38.892 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:38.892 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:38.892 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:39.321 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:39.811 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:39.812 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:39.813 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:39.813 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:39.813 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:39.814 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:40.322 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:41.252 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:41.252 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:41.252 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:41.253 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:41.253 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:41.253 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:41.323 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:42.317 7f9507e93400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:42:42.319 7f9507e93400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:42:42.319 7f9507e93400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:42:42.319 7f9507e93400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:42.319 7f9507e93400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:42:42.319 7f9507e93400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:42.319 7f9507e93400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:42:42.320 7f9507e93400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:42:42.320 7f9507e93400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:42:42.320 7f9507e93400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:42.320 7f9507e93400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:42.320 7f9507e93900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:42:42.321 7f9507e93900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:42:42.324 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:43.325 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:44.326 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:44.526 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:44.527 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:44.527 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:44.527 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:44.527 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:44.528 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:45.327 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:46.329 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:47.332 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:48.333 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:49.335 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:50.335 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:51.337 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:52.210 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:52.211 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:52.211 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:52.211 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:52.211 7f9508577a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:52.212 7f9508577a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:52.338 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:53.339 7f95085c7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:53.423 7f9508577a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:42:53.423 7f9508577a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:42:53.424 7f9508577a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:53.526 7f95085c77c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:42:53.526 7f950d051c00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:42:53.526 7f950d051c00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:42:53.526 7f9508577680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:42:53.526 7f9508577680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:42:53.546 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:42:53.547 7f950d051c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:43:03.710 7f11fce05d80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:43:03.710 7f11fce05d80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:43:03.710 7f11fce05d80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:43:03.713 7f11fce05d80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:43:03.714 7f11f83812c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:43:03.714 7f11f83812c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:43:03.714 7f11f83817c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:43:03.714 7f11f83817c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:43:03.715 7f11f83817c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:43:03.715 7f11f83817c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:43:03.715 7f11f83817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.715 7f11f83817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:43:03.715 7f11f83817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.715 7f11f83817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.716 7f11f83817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353783715202 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:43:03.716 7f11f83817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.716 7f11f83817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.716 7f11f83817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:03.717 7f11f83817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.717 7f11f83817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.717 7f11f83817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:03.717 7f11f83817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.717 7f11f83817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.717 7f11f83817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:03.717 7f11f83817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.717 7f11f83817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.717 7f11f83817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:03.717 7f11f83817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.718 7f11f8381e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:43:03.718 7f11f8381e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:43:03.719 7f11f8381c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:43:03.720 7f11f8331540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.720 7f11f8331540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:43:03.720 7f11f8331540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.720 7f11f8331540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.720 7f11f8331540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:03.720 7f11f8331540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.778 7f11f83817c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:43:03.785 7f11f83817c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:43:03.786 7f11f82c90e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.786 7f11f82c90e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:03.786 7f11f82c90e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.786 7f11f82c90e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.786 7f11f82c90e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:03.786 7f11f82c90e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.788 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:03.789 7f11f83317c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.789 7f11f83317c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:03.790 7f11f83317c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.790 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:03.790 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:03.791 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:03.791 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:03.792 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:03.793 7f11f83317c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.793 7f11f83317c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:03.793 7f11f83317c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.794 7f11f83817c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:43:03.794 7f11f83317c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.794 7f11f83317c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:03.794 7f11f83317c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.796 7f11f82c90e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.796 7f11f82c90e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:03.796 7f11f82c90e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.846 7f11f83817c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:43:03.846 7f11f82c9c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.847 7f11f82c9c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:03.847 7f11f82c9c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.850 7f11f83817c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:43:03.851 7f11f82c9c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.851 7f11f82c9c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:03.851 7f11f82c9c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.852 7f11f83817c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:43:03.852 7f11f83817c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:43:03.852 7f11f83817c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:43:03.853 7f11f82c9f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:43:03.853 7f11f81ff220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.854 7f11f81ff220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:03.854 7f11f81ff220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.854 7f11f81ff220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.854 7f11f81ff220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:03.854 7f11f81ff220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.856 7f11f83817c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:43:03.859 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:03.860 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:03.860 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:03.861 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:03.861 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:03.861 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:03.863 7f11f82c9f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:43:03.863 7f11f81ff4a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.864 7f11f81ff4a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:03.864 7f11f81ff4a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.888 7f11f83817c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:43:03.892 7f11f81ff900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:03.892 7f11f81ff900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:43:03.892 7f11f81ff900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:04.027 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:04.027 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:04.028 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:04.028 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:04.028 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:04.029 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:04.364 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:04.365 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:04.365 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:04.366 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:04.366 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:04.367 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:04.719 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:05.000 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:05.001 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:05.001 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:05.002 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:05.002 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:05.002 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:05.721 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:06.497 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:06.497 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:06.498 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:06.498 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:06.498 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:06.499 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:06.722 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:07.715 7f11f81ff900 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:43:07.716 7f11f81ff900 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:43:07.716 7f11f81ff900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:43:07.716 7f11f81ff900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:07.716 7f11f81ff900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:43:07.716 7f11f81ff900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:07.716 7f11f81ff900 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:43:07.717 7f11f81ff900 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:43:07.717 7f11f81ff900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:43:07.717 7f11f81ff900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:07.717 7f11f81ff900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:07.717 7f11f9a06360 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:43:07.718 7f11f9a06360 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:43:07.723 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:08.725 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:09.094 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:09.095 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:09.095 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:09.096 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:09.096 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:09.096 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:09.726 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:10.727 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:11.728 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:12.730 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:13.731 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:14.732 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:15.069 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:15.070 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:15.070 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:15.071 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:15.071 7f11f8331a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:15.072 7f11f8331a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:15.733 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:16.734 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:17.735 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:18.736 7f11f8381f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:18.787 7f11f8331a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:43:18.787 7f11f8331a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:43:18.788 7f11f8331a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:18.961 7f11f83817c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:43:18.961 7f11fce05d80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:43:18.961 7f11fce05d80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:43:18.961 7f11f83319a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:43:18.961 7f11f83319a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:43:18.976 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:43:18.976 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:43:18.976 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:43:18.976 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:43:18.976 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:43:18.976 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:43:18.976 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:43:18.976 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:43:18.976 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:43:18.977 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:43:18.977 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:43:18.977 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:43:18.977 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:43:18.977 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:43:18.977 7f11fce05d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:43:28.263 7f05d9be0740 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:43:28.263 7f05d9be0740 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:43:28.263 7f05d9be0740 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:43:28.268 7f05d9be0740 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:43:28.269 7f05d514c2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:43:28.269 7f05d514c2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:43:28.269 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:43:28.270 7f05d514c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:43:28.270 7f05d514c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:43:28.271 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:43:28.271 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.271 7f05d514c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:43:28.271 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.271 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.271 7f05d514c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353808270879 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:43:28.271 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.272 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.273 7f05d514c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:28.273 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.273 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.274 7f05d514c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:28.274 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.274 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.274 7f05d514c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:28.274 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.274 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.274 7f05d514c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:28.274 7f05d514c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.275 7f05d514cf40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:43:28.275 7f05d514cf40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:43:28.277 7f05d514ce00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:43:28.277 7f05d50fc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.277 7f05d50fc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:43:28.278 7f05d50fc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.278 7f05d50fc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.278 7f05d50fc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:28.278 7f05d50fc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.345 7f05d514c7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:43:28.353 7f05d514c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:43:28.354 7f05d50960e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.354 7f05d50960e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:28.354 7f05d50960e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.354 7f05d50960e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.355 7f05d50960e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:28.355 7f05d50960e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.357 7f05d50fc9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.357 7f05d50fc9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:28.357 7f05d50fc9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.358 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:28.359 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:28.359 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:28.360 7f05d514c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:43:28.360 7f05d50fc9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.361 7f05d50fc9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:28.361 7f05d50fc9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.369 7f05d5096d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.370 7f05d5096d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:28.370 7f05d5096d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.370 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:28.370 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:28.371 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:28.373 7f05d5096d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.373 7f05d5096d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:28.373 7f05d5096d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.408 7f05d514c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:43:28.409 7f05d5096c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.410 7f05d5096c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:28.410 7f05d5096c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.413 7f05d514c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:43:28.413 7f05d5096c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.414 7f05d5096c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:28.414 7f05d5096c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.414 7f05d514c7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:43:28.414 7f05d514c7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:43:28.414 7f05d514c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:43:28.414 7f05d5096f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:43:28.415 7f05d4fd1220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.415 7f05d4fd1220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:28.415 7f05d4fd1220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.416 7f05d4fd1220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.416 7f05d4fd1220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:28.416 7f05d4fd1220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.419 7f05d514c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:43:28.421 7f05d5096f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:43:28.421 7f05d4fd19a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.422 7f05d4fd19a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:28.422 7f05d4fd19a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.445 7f05d514c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:43:28.447 7f05d67de180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.447 7f05d67de180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:43:28.448 7f05d67de180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:28.501 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:28.502 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:28.502 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:28.503 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:28.503 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:28.503 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:28.819 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:28.820 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:28.820 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:28.821 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:28.821 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:28.822 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:29.276 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:29.370 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:29.371 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:29.371 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:29.371 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:29.371 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:29.372 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:30.277 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:30.601 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:30.602 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:30.602 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:30.602 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:30.603 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:30.603 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:31.278 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:32.094 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:32.095 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:32.095 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:32.096 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:32.096 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:32.096 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:32.270 7f05d67de680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:43:32.272 7f05d67de680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:43:32.272 7f05d67de680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:43:32.272 7f05d67de680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:32.272 7f05d67de680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:43:32.272 7f05d67de680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:32.272 7f05d67de680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:43:32.273 7f05d67de680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:43:32.273 7f05d67de680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:43:32.273 7f05d67de680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:32.273 7f05d67de680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:32.273 7f05d67de900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:43:32.274 7f05d67de900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:43:32.279 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:33.281 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:34.282 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:35.283 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:36.244 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:36.244 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:36.244 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:36.245 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:36.245 7f05d50fcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:36.245 7f05d50fcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:36.284 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:37.285 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:38.286 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:39.287 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:40.287 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:41.292 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:42.294 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:43.281 7f05d50fcc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:43:43.281 7f05d50fcc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:43:43.282 7f05d50fcc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:43.295 7f05d50fc2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:43.537 7f05d514c7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:43:43.537 7f05d9be0740 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:43:43.538 7f05d9be0740 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:43:43.538 7f05d50fc860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:43:43.538 7f05d50fc860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:43:43.553 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:43:43.553 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:43:43.553 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:43:43.553 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:43:43.553 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:43:43.553 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:43:43.553 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:43:43.553 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:43:43.554 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:43:43.554 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:43:43.554 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:43:43.554 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:43:43.554 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:43:43.554 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:43:43.554 7f05d9be0740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:43:52.833 7fba31040ec0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:43:52.833 7fba31040ec0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:43:52.833 7fba31040ec0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:43:52.837 7fba31040ec0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:43:52.838 7fba2c2712c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:43:52.838 7fba2c2712c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:43:52.838 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:43:52.839 7fba2c2717c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:43:52.839 7fba2c2717c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:43:52.839 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:43:52.839 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.839 7fba2c2717c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:43:52.839 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.839 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.839 7fba2c2717c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353832839240 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:43:52.839 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.840 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.840 7fba2c2717c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:52.840 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.840 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.841 7fba2c2717c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:52.841 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.841 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.841 7fba2c2717c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:52.841 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.841 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.841 7fba2c2717c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:52.841 7fba2c2717c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.841 7fba2c271f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:43:52.841 7fba2c271f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:43:52.842 7fba2c271e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:43:52.843 7fba2c2207c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.843 7fba2c2207c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:43:52.843 7fba2c2207c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.843 7fba2c2207c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.843 7fba2c2207c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:52.843 7fba2c2207c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.907 7fba2c2717c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:43:52.916 7fba2c2717c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:43:52.917 7fba2c1b40e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.917 7fba2c1b40e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:52.918 7fba2c1b40e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.918 7fba2c1b40e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.918 7fba2c1b40e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:52.918 7fba2c1b40e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.919 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:52.920 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:52.920 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:52.921 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:52.921 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:52.922 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:52.925 7fba2c2717c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:43:52.927 7fba2c2209a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.927 7fba2c2209a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:52.927 7fba2c2209a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.928 7fba2c2209a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.928 7fba2c2209a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:52.928 7fba2c2209a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.937 7fba2c1b4d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.938 7fba2c1b4d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:52.938 7fba2c1b4d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.939 7fba2c1b4d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.940 7fba2c1b4d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:52.940 7fba2c1b4d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.968 7fba2c2717c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:43:52.969 7fba2c1b4c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.969 7fba2c1b4c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:52.970 7fba2c1b4c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.972 7fba2c2717c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:43:52.972 7fba2c1b4c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.973 7fba2c1b4c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:52.973 7fba2c1b4c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.973 7fba2c2717c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:43:52.973 7fba2c2717c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:43:52.973 7fba2c2717c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:43:52.974 7fba2c1b4f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:43:52.974 7fba2c0f5220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.974 7fba2c0f5220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:52.975 7fba2c0f5220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.975 7fba2c0f5220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.975 7fba2c0f5220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:52.975 7fba2c0f5220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.979 7fba2c2717c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:43:52.980 7fba2c1b4f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:43:52.981 7fba2c0f59a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:52.981 7fba2c0f59a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:52.981 7fba2c0f59a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:53.004 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:53.004 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:53.004 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:53.005 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:53.005 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:53.005 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:53.029 7fba2c2717c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:43:53.032 7fba2c0f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:53.032 7fba2c0f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:43:53.032 7fba2c0f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:53.221 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:53.222 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:53.222 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:53.223 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:53.223 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:53.223 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:53.729 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:53.730 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:53.730 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:53.731 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:53.731 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:53.731 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:53.843 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:54.812 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:54.812 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:54.813 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:54.813 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:54.813 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:54.814 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:54.844 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:55.845 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:56.839 7fba2c0f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:43:56.840 7fba2c0f57c0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:43:56.840 7fba2c0f57c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:43:56.840 7fba2c0f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:56.840 7fba2c0f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:43:56.841 7fba2c0f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:56.841 7fba2c0f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:43:56.841 7fba2c0f57c0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:43:56.841 7fba2c0f57c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:43:56.841 7fba2c0f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:56.841 7fba2c0f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:56.842 7fba2d8f7180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:43:56.843 7fba2d8f7180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:43:56.846 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:56.902 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:56.902 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:56.903 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:56.903 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:56.903 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:56.903 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:57.847 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:58.848 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:59.849 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:00.260 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:00.260 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:00.260 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:00.261 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:00.261 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:00.261 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:00.851 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:01.852 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:02.853 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:03.854 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:04.303 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:04.304 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:04.305 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:04.305 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:04.305 7fba2c220c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:04.306 7fba2c220c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:04.855 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:05.857 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:06.858 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:07.859 7fba2c2202c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:07.923 7fba2c220c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:44:07.923 7fba2c220c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:44:07.924 7fba2c220c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:08.039 7fba2c2717c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:44:08.040 7fba31040ec0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:44:08.040 7fba31040ec0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:44:08.040 7fba2c220b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:44:08.040 7fba2c220b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:44:08.053 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:44:08.053 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:44:08.055 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:44:08.055 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:44:08.055 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:44:08.055 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:44:08.055 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:44:08.055 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:44:08.055 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:44:08.055 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:44:08.055 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:44:08.056 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:44:08.056 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:44:08.056 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:44:08.056 7fba31040ec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:44:16.865 7f480527bc00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:44:16.865 7f480527bc00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:44:16.865 7f480527bc00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:44:16.868 7f480527bc00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:44:16.869 7f48007e82c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:44:16.869 7f48007e82c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:44:16.869 7f48007e87c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:44:16.870 7f48007e87c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:44:16.870 7f48007e87c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:44:16.870 7f48007e87c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:44:16.870 7f48007e87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.871 7f48007e87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:44:16.871 7f48007e87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.871 7f48007e87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.871 7f48007e87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353856870591 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:44:16.871 7f48007e87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.871 7f48007e87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.872 7f48007e87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:44:16.872 7f48007e87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.872 7f48007e87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.872 7f48007e87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:44:16.872 7f48007e87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.872 7f48007e87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.872 7f48007e87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:44:16.872 7f48007e87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.872 7f48007e87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.872 7f48007e87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:16.872 7f48007e87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.873 7f48007e8e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:44:16.873 7f48007e8e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:44:16.874 7f48007e8c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:44:16.874 7f4800798540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.874 7f4800798540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:44:16.874 7f4800798540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.874 7f4800798540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.874 7f4800798540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:16.875 7f4800798540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.933 7f48007e87c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:44:16.944 7f48007e87c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:44:16.945 7f48006f7040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.945 7f48006f7040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:16.945 7f48006f7040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.945 7f48006f7040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.945 7f48006f7040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:16.945 7f48006f7040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.946 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:16.948 7f48007987c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.948 7f48007987c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:16.948 7f48007987c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.948 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:16.949 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:16.949 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:16.950 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:16.951 7f48007e87c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:44:16.951 7f48007987c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.951 7f48007987c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:16.951 7f48007987c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.951 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:16.960 7f48006f7c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.960 7f48006f7c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:16.960 7f48006f7c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.962 7f48006f7c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:16.962 7f48006f7c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:16.962 7f48006f7c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:17.017 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:17.018 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:17.018 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:17.018 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:17.018 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:17.019 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:17.032 7f48007e87c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:44:17.033 7f48006f7ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:17.033 7f48006f7ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:17.033 7f48006f7ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:17.038 7f48007e87c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:44:17.039 7f48007e87c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:44:17.039 7f48007e87c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:44:17.039 7f48007e87c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:44:17.040 7f48006f7ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:17.040 7f48006f7ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:17.040 7f48006f7ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:17.040 7f48006f7ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:17.040 7f48006f7ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:44:17.040 7f48006f7ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:17.041 7f48006f7e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:44:17.044 7f4800662220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:17.044 7f4800662220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:44:17.044 7f4800662220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:17.049 7f48007e87c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:44:17.051 7f48006f7e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:44:17.052 7f48006624a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:17.053 7f48006624a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:44:17.053 7f48006624a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:17.110 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:17.111 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:17.111 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:17.111 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:17.111 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:17.112 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:17.130 7f48007e87c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:44:17.132 7f4800662ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:17.133 7f4800662ea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:44:17.133 7f4800662ea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:17.301 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:17.301 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:17.301 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:17.302 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:17.302 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:17.303 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:17.824 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:17.825 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:17.825 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:17.826 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:17.826 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:17.826 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:17.874 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:18.561 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:18.561 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:18.561 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:18.562 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:18.562 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:18.563 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:18.875 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:19.590 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:19.590 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:19.590 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:19.591 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:19.591 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:19.592 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:19.876 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:20.870 7f4800662ea0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:44:20.871 7f4800662ea0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:44:20.871 7f4800662ea0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:44:20.872 7f4800662ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:44:20.872 7f4800662ea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:44:20.872 7f4800662ea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:44:20.872 7f4800662ea0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:44:20.872 7f4800662ea0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:44:20.873 7f4800662ea0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:44:20.873 7f4800662ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:20.873 7f4800662ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:20.873 7f48028ea180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:44:20.874 7f48028ea180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:44:20.877 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:20.880 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:20.881 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:20.881 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:20.881 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:20.881 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:20.882 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:21.878 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:22.585 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:22.586 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:22.586 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:22.587 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:22.587 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:22.587 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:22.880 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:23.881 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:24.707 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:24.708 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:24.708 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:24.709 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:24.709 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:24.709 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:24.882 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:25.883 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:26.885 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:27.887 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:28.888 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:29.849 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:29.851 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:29.851 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:29.851 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:29.851 7f4800798a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:29.852 7f4800798a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:29.890 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:30.891 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:31.892 7f48007e8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:31.962 7f4800798a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:44:31.963 7f4800798a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:44:31.963 7f4800798a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:32.166 7f48007e87c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:44:32.166 7f480527bc00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:44:32.166 7f480527bc00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:44:32.166 7f48007989a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:44:32.166 7f48007989a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:44:32.180 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:44:32.180 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:44:32.180 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:44:32.180 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:44:32.180 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:44:32.180 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:44:32.180 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:44:32.180 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:44:32.180 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:44:32.180 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:44:32.181 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:44:32.181 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:44:32.181 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:44:32.181 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:44:32.181 7f480527bc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:44:41.180 7f63fd90a340 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:44:41.184 7f63fd90a340 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:44:41.184 7f63fd90a340 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:44:41.193 7f63fd90a340 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:44:41.194 7f63f8e512c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:44:41.194 7f63f8e512c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:44:41.194 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:44:41.195 7f63f8e517c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:44:41.200 7f63f8e517c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:44:41.200 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:44:41.200 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.201 7f63f8e517c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:44:41.201 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.201 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.201 7f63f8e517c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353881200100 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:44:41.201 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.202 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.202 7f63f8e517c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:44:41.202 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.202 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.202 7f63f8e517c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:44:41.202 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.202 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.203 7f63f8e517c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:44:41.203 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.203 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.203 7f63f8e517c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:41.203 7f63f8e517c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.203 7f63f8e51e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:44:41.204 7f63f8e51e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:44:41.205 7f63f8e51c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:44:41.206 7f63f8e01540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.206 7f63f8e01540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:44:41.206 7f63f8e01540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.206 7f63f8e01540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.207 7f63f8e01540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:41.207 7f63f8e01540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.313 7f63f8e517c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:44:41.334 7f63f8e517c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:44:41.335 7f63f8d9d040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.335 7f63f8d9d040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:41.335 7f63f8d9d040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.336 7f63f8d9d040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.336 7f63f8d9d040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:41.336 7f63f8d9d040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.337 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:41.339 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:41.339 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:41.341 7f63f8e017c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.341 7f63f8e017c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:41.341 7f63f8e017c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.341 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:41.342 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:41.342 7f63f8e517c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:44:41.343 7f63f8e017c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.344 7f63f8e017c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:41.360 7f63f8e017c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.369 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:41.392 7f63f8d9dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.392 7f63f8d9dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:41.392 7f63f8d9dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.400 7f63f8d9dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.400 7f63f8d9dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:41.401 7f63f8d9dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.420 7f63f8e517c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:44:41.421 7f63f8d9dae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.421 7f63f8d9dae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:41.422 7f63f8d9dae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.425 7f63f8e517c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:44:41.425 7f63f8d9dae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.426 7f63f8d9dae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:41.426 7f63f8d9dae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.426 7f63f8e517c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:44:41.427 7f63f8e517c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:44:41.427 7f63f8e517c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:44:41.427 7f63f8d9de00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:44:41.428 7f63f8ccc0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.428 7f63f8ccc0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:44:41.428 7f63f8ccc0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.428 7f63f8ccc0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.429 7f63f8ccc0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:44:41.429 7f63f8ccc0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.432 7f63f8e517c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:44:41.440 7f63f8d9de00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:44:41.440 7f63f8cccae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.440 7f63f8cccae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:44:41.441 7f63f8cccae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.481 7f63f8e517c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:44:41.488 7f63fa4e8400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.489 7f63fa4e8400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:44:41.490 7f63fa4e8400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:41.499 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:41.502 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:41.502 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:41.503 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:41.503 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:41.506 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:41.795 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:41.796 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:41.796 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:41.797 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:41.797 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:41.797 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:42.205 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:42.585 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:42.586 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:42.586 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:42.587 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:42.587 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:42.587 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:43.206 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:44.207 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:44.773 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:44.774 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:44.774 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:44.774 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:44.774 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:44.775 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:45.195 7f63fa4e8400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:44:45.217 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:45.218 7f63fa4e8400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:44:45.218 7f63fa4e8400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:44:45.219 7f63fa4e8400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:44:45.219 7f63fa4e8400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:44:45.219 7f63fa4e8400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:44:45.219 7f63fa4e8400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:44:45.219 7f63fa4e8400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:44:45.220 7f63fa4e8400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:44:45.220 7f63fa4e8400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:45.220 7f63fa4e8400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:45.220 7f63fa4e8900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:44:45.221 7f63fa4e8900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:44:46.219 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:47.220 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:48.221 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:48.820 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:48.821 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:48.821 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:48.821 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:48.821 7f63f8e01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:48.822 7f63f8e01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:49.222 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:50.223 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:51.225 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:52.226 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:53.227 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:54.228 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:55.229 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:56.230 7f63f8e51f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:56.263 7f63f8e01a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:44:56.263 7f63f8e01a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:44:56.264 7f63f8e01a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:56.550 7f63f8e517c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:44:56.551 7f63fd90a340 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:44:56.551 7f63fd90a340 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:44:56.551 7f63f8e01680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:44:56.551 7f63f8e01680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:44:56.565 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:44:56.565 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:44:56.565 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:44:56.565 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:44:56.565 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:44:56.566 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:44:56.566 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:44:56.566 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:44:56.566 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:44:56.566 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:44:56.566 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:44:56.566 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:44:56.566 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:44:56.566 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:44:56.566 7f63fd90a340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:45:05.958 7fa402101e00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:45:05.958 7fa402101e00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:45:05.958 7fa402101e00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:45:05.962 7fa402101e00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:45:05.962 7fa3fd6702c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:45:05.962 7fa3fd6702c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:45:05.963 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:45:05.963 7fa3fd6707c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:45:05.963 7fa3fd6707c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:45:05.963 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:45:05.964 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.964 7fa3fd6707c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:45:05.964 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.964 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.964 7fa3fd6707c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353905963774 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:45:05.964 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.965 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.965 7fa3fd6707c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:05.965 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.965 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.965 7fa3fd6707c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:05.965 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.965 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.965 7fa3fd6707c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:05.965 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.965 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.965 7fa3fd6707c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:05.965 7fa3fd6707c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.966 7fa3fd670e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:45:05.966 7fa3fd670e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:45:05.967 7fa3fd670c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:45:05.968 7fa3fd620540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.968 7fa3fd620540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:45:05.968 7fa3fd620540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.968 7fa3fd620540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:05.968 7fa3fd620540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:05.968 7fa3fd620540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.035 7fa3fd6707c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:45:06.061 7fa3fd6707c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:45:06.062 7fa3fd5bd040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.062 7fa3fd5bd040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:06.062 7fa3fd5bd040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.062 7fa3fd5bd040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.062 7fa3fd5bd040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:06.062 7fa3fd5bd040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.063 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:06.064 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.064 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:06.066 7fa3fd6207c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.066 7fa3fd6207c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:06.066 7fa3fd6207c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.066 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.066 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:06.068 7fa3fd6707c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:45:06.068 7fa3fd6207c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.068 7fa3fd6207c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:06.068 7fa3fd6207c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.068 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.076 7fa3fd5bdc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.077 7fa3fd5bdc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:06.077 7fa3fd5bdc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.078 7fa3fd5bdc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.079 7fa3fd5bdc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:06.079 7fa3fd5bdc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.104 7fa3fd6707c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:45:06.105 7fa3fd5bdae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.105 7fa3fd5bdae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:06.105 7fa3fd5bdae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.108 7fa3fd6707c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:45:06.109 7fa3fd5bdae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.109 7fa3fd5bdae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:06.109 7fa3fd5bdae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.110 7fa3fd6707c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:45:06.110 7fa3fd6707c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:45:06.110 7fa3fd6707c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:45:06.111 7fa3fd5bde00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:45:06.111 7fa3fd4f50e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.112 7fa3fd4f50e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:06.113 7fa3fd4f50e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.113 7fa3fd4f50e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.113 7fa3fd4f50e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:06.113 7fa3fd4f50e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.120 7fa3fd6707c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:45:06.122 7fa3fd5bde00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:45:06.122 7fa3fd4f5860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.123 7fa3fd4f5860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:06.123 7fa3fd4f5860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.156 7fa3fd6707c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:45:06.158 7fa3fec38180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.158 7fa3fec38180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:45:06.159 7fa3fec38180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:06.209 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:06.210 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.210 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:06.210 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.211 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:06.211 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.502 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:06.503 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.503 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:06.504 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.504 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:06.505 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.861 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:06.862 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.862 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:06.863 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.863 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:06.864 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.967 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:07.837 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:07.838 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:07.839 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:07.839 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:07.839 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:07.840 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:07.968 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:08.969 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:09.964 7fa3fec38680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:45:09.965 7fa3fec38680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:45:09.965 7fa3fec38680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:45:09.965 7fa3fec38680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:09.965 7fa3fec38680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:45:09.965 7fa3fec38680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:09.965 7fa3fec38680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:45:09.966 7fa3fec38680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:45:09.966 7fa3fec38680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:45:09.966 7fa3fec38680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:09.966 7fa3fec38680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:09.966 7fa3fec38900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:45:09.967 7fa3fec38900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:45:09.970 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:10.385 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:10.385 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:10.385 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:10.386 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:10.386 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:10.387 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:10.971 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:11.972 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:12.973 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:13.995 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:14.995 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:15.684 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:15.685 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:15.685 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:15.685 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:15.685 7fa3fd620a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:15.686 7fa3fd620a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:15.996 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:16.997 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:17.999 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:18.999 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:20.001 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:21.002 7fa3fd670f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:21.010 7fa3fd620a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:45:21.011 7fa3fd620a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:45:21.011 7fa3fd620a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:21.185 7fa3fd6707c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:45:21.185 7fa402101e00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:45:21.186 7fa402101e00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:45:21.186 7fa3fd620860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:45:21.186 7fa3fd620860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:45:21.201 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:45:21.201 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:45:21.201 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:45:21.201 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:45:21.201 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:45:21.202 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:45:21.202 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:45:21.202 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:45:21.202 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:45:21.202 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:45:21.202 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:45:21.202 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:45:21.202 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:45:21.202 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:45:21.202 7fa402101e00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:45:30.454 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:45:30.454 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:45:30.454 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:45:30.460 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:45:30.461 7f4fc9d752c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:45:30.461 7f4fc9d752c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:45:30.461 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:45:30.462 7f4fc9d757c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:45:30.462 7f4fc9d757c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:45:30.462 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:45:30.462 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.463 7f4fc9d757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:45:30.463 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.463 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.463 7f4fc9d757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353930462516 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:45:30.463 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.464 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.464 7f4fc9d757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:30.464 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.465 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.465 7f4fc9d757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:30.465 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.465 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.465 7f4fc9d757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:30.465 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.465 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.465 7f4fc9d757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:30.465 7f4fc9d757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.466 7f4fc9d75e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:45:30.466 7f4fc9d75e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:45:30.468 7f4fc9d75c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:45:30.468 7f4fc9d25540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.468 7f4fc9d25540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:45:30.468 7f4fc9d25540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.468 7f4fc9d25540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.469 7f4fc9d25540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:30.469 7f4fc9d25540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.540 7f4fc9d757c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:45:30.552 7f4fc9d757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:45:30.553 7f4fc9cda040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.553 7f4fc9cda040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:30.553 7f4fc9cda040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.553 7f4fc9cda040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.553 7f4fc9cda040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:30.553 7f4fc9cda040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.555 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:30.556 7f4fc9d257c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.557 7f4fc9d257c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:30.557 7f4fc9d257c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.557 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:30.557 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:30.558 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:30.558 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:30.558 7f4fc9d757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:45:30.559 7f4fc9d257c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.559 7f4fc9d257c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:30.559 7f4fc9d257c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.569 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:30.571 7f4fc9cdac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.572 7f4fc9cdac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:30.572 7f4fc9cdac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.576 7f4fc9cdac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.577 7f4fc9cdac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:30.578 7f4fc9cdac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.605 7f4fc9d757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:45:30.605 7f4fc9cdaae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.606 7f4fc9cdaae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:30.606 7f4fc9cdaae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.609 7f4fc9d757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:45:30.609 7f4fc9cdaae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.609 7f4fc9cdaae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:30.609 7f4fc9cdaae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.610 7f4fc9d757c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:45:30.610 7f4fc9d757c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:45:30.611 7f4fc9d757c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:45:30.611 7f4fc9cdae00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:45:30.611 7f4fcb34c0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.612 7f4fcb34c0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:30.612 7f4fcb34c0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.612 7f4fcb34c0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.612 7f4fcb34c0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:30.612 7f4fcb34c0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.623 7f4fc9d757c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:45:30.629 7f4fc9cdae00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:45:30.630 7f4fcb34c860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.630 7f4fcb34c860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:30.630 7f4fcb34c860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.660 7f4fc9d757c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:45:30.663 7f4fcb335400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.663 7f4fcb335400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:45:30.663 7f4fcb335400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:30.708 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:30.709 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:30.709 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:30.709 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:30.709 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:30.710 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:31.084 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:31.085 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:31.085 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:31.086 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:31.086 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:31.086 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:31.468 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:31.995 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:31.996 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:31.997 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:31.997 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:31.997 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:31.998 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:32.469 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:33.153 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:33.154 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:33.154 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:33.154 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:33.155 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:33.155 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:33.470 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:34.462 7f4fcb335400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:45:34.463 7f4fcb335400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:45:34.464 7f4fcb335400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:45:34.464 7f4fcb335400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:34.464 7f4fcb335400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:45:34.464 7f4fcb335400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:34.464 7f4fcb335400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:45:34.464 7f4fcb335400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:45:34.464 7f4fcb335400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:45:34.465 7f4fcb335400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:34.465 7f4fcb335400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:34.465 7f4fcb335900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:45:34.466 7f4fcb335900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:45:34.471 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:35.123 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:35.124 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:35.124 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:35.125 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:35.125 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:35.125 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:35.472 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:36.473 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:37.474 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:38.476 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:39.424 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:39.425 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:39.425 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:39.426 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:39.426 7f4fc9d25a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:39.426 7f4fc9d25a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:39.477 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:40.478 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:41.479 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:42.480 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:43.481 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:44.482 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:45.483 7f4fc9d75f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:45.559 7f4fc9d25a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:45:45.559 7f4fc9d25a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:45:45.560 7f4fc9d25a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:45.748 7f4fc9d757c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:45:45.748 7f4fce7ffc00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:45:45.748 7f4fce7ffc00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:45:45.749 7f4fc9d25860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:45:45.749 7f4fc9d25860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:45:45.771 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:45:45.771 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:45:45.772 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:45:45.772 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:45:45.772 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:45:45.772 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:45:45.772 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:45:45.772 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:45:45.772 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:45:45.772 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:45:45.772 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:45:45.772 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:45:45.772 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:45:45.773 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:45:45.773 7f4fce7ffc00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:45:55.949 7fd950ff0d00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:45:55.949 7fd950ff0d00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:45:55.949 7fd950ff0d00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:45:55.953 7fd950ff0d00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:45:55.953 7fd94c56e2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:45:55.953 7fd94c56e2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:45:55.953 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:45:55.954 7fd94c56e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:45:55.954 7fd94c56e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:45:55.955 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:45:55.955 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.955 7fd94c56e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:45:55.955 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.955 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.955 7fd94c56e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353955954748 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:45:55.955 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.959 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.959 7fd94c56e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:55.959 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.959 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.959 7fd94c56e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:55.959 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.959 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.959 7fd94c56e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:55.959 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.959 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.960 7fd94c56e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:55.960 7fd94c56e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.960 7fd94c56ee00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:45:55.960 7fd94c56ee00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:45:55.962 7fd94c56ec20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:45:55.974 7fd94c51e540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.974 7fd94c51e540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:45:55.974 7fd94c51e540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.974 7fd94c51e540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:55.974 7fd94c51e540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:55.975 7fd94c51e540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.050 7fd94c56e7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:45:56.065 7fd94c56e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:45:56.066 7fd94e67f040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.066 7fd94e67f040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:56.066 7fd94e67f040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.066 7fd94e67f040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.066 7fd94e67f040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:56.066 7fd94e67f040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.069 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:56.070 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:56.070 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:56.071 7fd94c56e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:45:56.071 7fd94c51e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.072 7fd94c51e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:56.072 7fd94c51e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.072 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:56.072 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:56.086 7fd94c51e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.087 7fd94c51e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:56.087 7fd94c51e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.093 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:56.095 7fd94e67fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.095 7fd94e67fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:56.095 7fd94e67fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.097 7fd94e67fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.097 7fd94e67fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:56.097 7fd94e67fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.123 7fd94c56e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:45:56.124 7fd94e67fae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.124 7fd94e67fae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:56.124 7fd94e67fae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.128 7fd94c56e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:45:56.128 7fd94e67fae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.128 7fd94e67fae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:56.128 7fd94e67fae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.129 7fd94c56e7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:45:56.129 7fd94c56e7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:45:56.129 7fd94c56e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:45:56.130 7fd94e67fe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:45:56.130 7fd94db480e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.130 7fd94db480e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:56.130 7fd94db480e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.130 7fd94db480e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.131 7fd94db480e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:56.131 7fd94db480e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.133 7fd94c56e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:45:56.151 7fd94e67fe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:45:56.152 7fd94db48d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.152 7fd94db48d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:56.152 7fd94db48d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.186 7fd94c56e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:45:56.187 7fd94db48900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.188 7fd94db48900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:45:56.188 7fd94db48900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:56.200 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:56.200 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:56.200 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:56.202 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:56.202 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:56.202 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:56.484 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:56.485 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:56.485 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:56.488 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:56.488 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:56.488 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:56.962 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:57.064 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:57.065 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:57.065 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:57.066 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:57.066 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:57.066 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:57.963 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:57.965 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:57.966 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:57.966 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:57.966 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:57.966 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:57.967 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:58.964 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:59.777 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:59.778 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:59.778 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:59.778 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:59.778 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:59.779 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:59.955 7fd94db48900 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:45:59.956 7fd94db48900 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:45:59.956 7fd94db48900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:45:59.957 7fd94db48900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:59.957 7fd94db48900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:45:59.957 7fd94db48900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:59.957 7fd94db48900 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:45:59.957 7fd94db48900 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:45:59.957 7fd94db48900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:45:59.958 7fd94db48900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:59.958 7fd94db48900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:59.958 7fd94db21680 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:45:59.959 7fd94db21680 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:45:59.965 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:00.966 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:01.966 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:02.967 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:03.564 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:03.565 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:03.565 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:03.566 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:03.566 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:03.566 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:03.968 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:04.970 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:05.971 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:06.972 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:07.973 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:08.975 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:09.518 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:09.519 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:09.519 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:09.519 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:09.519 7fd94c51ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:09.520 7fd94c51ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:09.978 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:10.979 7fd94c56ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:11.025 7fd94c51ea40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:46:11.025 7fd94c51ea40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:46:11.026 7fd94c51ea40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:11.266 7fd94c56e7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:46:11.266 7fd950ff0d00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:46:11.267 7fd950ff0d00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:46:11.267 7fd94c51e860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:46:11.267 7fd94c51e860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:46:11.283 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:46:11.283 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:46:11.283 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:46:11.283 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:46:11.283 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:46:11.283 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:46:11.283 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:46:11.284 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:46:11.284 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:46:11.284 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:46:11.284 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:46:11.284 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:46:11.284 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:46:11.284 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:46:11.284 7fd950ff0d00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:46:21.561 7f5752d1af80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:46:21.562 7f5752d1af80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:46:21.562 7f5752d1af80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:46:21.565 7f5752d1af80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:46:21.566 7f574e2962c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:46:21.566 7f574e2962c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:46:21.566 7f574e2967c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:46:21.567 7f574e2967c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:46:21.567 7f574e2967c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:46:21.567 7f574e2967c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:46:21.567 7f574e2967c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.567 7f574e2967c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:46:21.567 7f574e2967c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.567 7f574e2967c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.567 7f574e2967c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353981567370 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:46:21.567 7f574e2967c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.568 7f574e2967c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.568 7f574e2967c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:46:21.568 7f574e2967c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.568 7f574e2967c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.569 7f574e2967c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:46:21.569 7f574e2967c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.569 7f574e2967c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.569 7f574e2967c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:46:21.569 7f574e2967c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.569 7f574e2967c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.569 7f574e2967c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:21.569 7f574e2967c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.569 7f574e296e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:46:21.569 7f574e296e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:46:21.571 7f574e296c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:46:21.571 7f574e246540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.572 7f574e246540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:46:21.572 7f574e246540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.572 7f574e246540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.572 7f574e246540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:21.572 7f574e246540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.643 7f574e2967c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:46:21.653 7f574e2967c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:46:21.654 7f574e1cd0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.654 7f574e1cd0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:21.654 7f574e1cd0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.655 7f574e1cd0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.655 7f574e1cd0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:21.655 7f574e1cd0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.657 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:21.658 7f574e2467c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.658 7f574e2467c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:21.658 7f574e2467c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.661 7f574e2967c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:46:21.671 7f574e2467c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.671 7f574e2467c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:21.671 7f574e2467c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.671 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:21.672 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:21.673 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:21.673 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:21.673 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:21.674 7f574e1cde00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.675 7f574e1cde00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:21.675 7f574e1cde00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.676 7f574e1cde00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.676 7f574e1cde00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:21.676 7f574e1cde00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.699 7f574e2967c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:46:21.701 7f574e2467c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.701 7f574e2467c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:21.701 7f574e2467c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.706 7f574e2967c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:46:21.706 7f574e2467c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.706 7f574e2467c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:21.706 7f574e2467c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.709 7f574e2967c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:46:21.710 7f574e2967c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:46:21.710 7f574e2967c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:46:21.711 7f574e2467c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:46:21.711 7f574e1a1220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.711 7f574e1a1220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:46:21.711 7f574e1a1220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.711 7f574e1a1220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.712 7f574e1a1220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:46:21.712 7f574e1a1220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.716 7f574e2967c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:46:21.723 7f574e2467c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:46:21.723 7f574e1a1c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.724 7f574e1a1c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:46:21.724 7f574e1a1c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.745 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:21.746 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:21.747 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:21.747 7f574e2967c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:46:21.748 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:21.748 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:21.749 7f574e122400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.750 7f574e122400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:46:21.750 7f574e122400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:21.751 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:21.921 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:21.922 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:21.922 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:21.923 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:21.923 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:21.923 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:22.350 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:22.351 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:22.351 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:22.351 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:22.352 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:22.352 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:22.571 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:23.250 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:23.251 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:23.251 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:23.252 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:23.252 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:23.253 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:23.572 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:24.574 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:25.363 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:25.364 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:25.364 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:25.365 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:25.365 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:25.365 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:25.567 7f574e122400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:46:25.569 7f574e122400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:46:25.569 7f574e122400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:46:25.569 7f574e122400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:25.569 7f574e122400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:46:25.569 7f574e122400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:25.569 7f574e122400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:46:25.569 7f574e122400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:46:25.569 7f574e122400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:46:25.570 7f574e122400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:25.570 7f574e122400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:25.570 7f574e122900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:46:25.571 7f574e122900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:46:25.574 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:26.575 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:27.576 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:28.577 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:29.030 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:29.032 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:29.032 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:29.032 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:29.033 7f574e246c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:29.033 7f574e246c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:29.579 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:30.580 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:31.581 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:32.582 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:33.583 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:34.584 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:35.585 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:36.587 7f574e296f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:36.667 7f574e246c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:46:36.668 7f574e246c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:46:36.668 7f574e246c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:36.818 7f574e2967c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:46:36.819 7f5752d1af80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:46:36.819 7f5752d1af80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:46:36.819 7f574e246680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:46:36.819 7f574e246680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:46:36.833 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:46:36.833 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:46:36.833 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:46:36.833 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:46:36.834 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:46:36.834 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:46:36.834 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:46:36.834 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:46:36.834 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:46:36.834 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:46:36.834 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:46:36.834 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:46:36.834 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:46:36.834 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:46:36.834 7f5752d1af80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:46:47.065 7fd4b4902f00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:46:47.066 7fd4b4902f00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:46:47.066 7fd4b4902f00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:46:47.070 7fd4b4902f00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:46:47.071 7fd4afe862c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:46:47.071 7fd4afe862c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:46:47.071 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:46:47.071 7fd4afe867c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:46:47.072 7fd4afe867c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:46:47.072 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:46:47.073 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.073 7fd4afe867c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:46:47.073 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.073 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.073 7fd4afe867c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789354007072375 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:46:47.073 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.074 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.074 7fd4afe867c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:46:47.074 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.074 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.075 7fd4afe867c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:46:47.075 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.075 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.075 7fd4afe867c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:46:47.075 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.075 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.075 7fd4afe867c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:47.075 7fd4afe867c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.075 7fd4afe86e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:46:47.075 7fd4afe86e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:46:47.077 7fd4afe86c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:46:47.077 7fd4afe35540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.078 7fd4afe35540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:46:47.078 7fd4afe35540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.078 7fd4afe35540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.078 7fd4afe35540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:47.078 7fd4afe35540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.138 7fd4afe867c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:46:47.145 7fd4afe867c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:46:47.147 7fd4afd1f0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.147 7fd4afd1f0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:47.147 7fd4afd1f0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.147 7fd4afd1f0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.147 7fd4afd1f0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:47.147 7fd4afd1f0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.149 7fd4afd1f0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.149 7fd4afd1f0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:47.149 7fd4afd1f0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.150 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:47.152 7fd4afe357c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.152 7fd4afe357c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:47.152 7fd4afe357c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.153 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:47.153 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:47.153 7fd4afe867c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:46:47.154 7fd4afe357c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.154 7fd4afe357c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:47.154 7fd4afe357c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.155 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:47.155 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:47.161 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:47.162 7fd4afd1fd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.162 7fd4afd1fd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:47.162 7fd4afd1fd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.184 7fd4afe867c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:46:47.185 7fd4afd1fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.185 7fd4afd1fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:47.185 7fd4afd1fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.188 7fd4afe867c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:46:47.189 7fd4afd1fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.189 7fd4afd1fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:47.189 7fd4afd1fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.189 7fd4afe867c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:46:47.189 7fd4afe867c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:46:47.189 7fd4afe867c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:46:47.190 7fd4afd1ff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:46:47.190 7fd4af781220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.191 7fd4af781220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:46:47.191 7fd4af781220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.191 7fd4af781220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.191 7fd4af781220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:46:47.191 7fd4af781220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.195 7fd4afe867c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:46:47.196 7fd4afd1ff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:46:47.196 7fd4af781720 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.196 7fd4af781720 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:46:47.196 7fd4af781720 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.249 7fd4afe867c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:46:47.251 7fd4af7817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.251 7fd4af7817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:46:47.251 7fd4af7817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:47.264 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:47.265 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:47.265 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:47.267 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:47.267 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:47.268 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:47.514 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:47.514 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:47.515 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:47.515 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:47.515 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:47.516 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:48.036 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:48.037 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:48.037 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:48.038 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:48.038 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:48.038 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:48.076 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:49.077 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:49.130 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:49.131 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:49.131 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:49.132 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:49.132 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:49.132 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:50.078 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:51.072 7fd4af7817c0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:46:51.073 7fd4af7817c0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:46:51.073 7fd4af7817c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:46:51.073 7fd4af7817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:51.074 7fd4af7817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:46:51.074 7fd4af7817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:51.074 7fd4af7817c0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:46:51.074 7fd4af7817c0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:46:51.074 7fd4af7817c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:46:51.074 7fd4af7817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:51.074 7fd4af7817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:51.075 7fd4b1f83180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:46:51.076 7fd4b1f83180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:46:51.079 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:51.880 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:51.881 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:51.881 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:51.881 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:51.881 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:51.882 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:52.080 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:53.081 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:54.082 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:55.083 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:56.084 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:57.085 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:57.251 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:57.252 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:57.252 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:57.253 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:57.253 7fd4afe35c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:57.253 7fd4afe35c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:58.086 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:59.087 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:00.088 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:01.089 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:02.082 7fd4afe35c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:47:02.082 7fd4afe35c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:47:02.083 7fd4afe35c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:02.091 7fd4afe86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:02.314 7fd4afe867c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:47:02.315 7fd4b4902f00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:47:02.315 7fd4b4902f00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:47:02.315 7fd4afe35b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:47:02.315 7fd4afe35b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:47:02.331 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:47:02.331 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:47:02.331 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:47:02.331 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:47:02.331 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:47:02.331 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:47:02.331 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:47:02.331 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:47:02.331 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:47:02.331 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:47:02.331 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:47:02.331 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:47:02.331 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:47:02.332 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:47:02.332 7fd4b4902f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:47:12.216 7f005a26eb40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:47:12.217 7f005a26eb40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:47:12.217 7f005a26eb40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:47:12.222 7f005a26eb40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:47:12.222 7f00552242c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:47:12.222 7f00552242c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:47:12.223 7f00552247c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:47:12.223 7f00552247c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:47:12.223 7f00552247c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:47:12.224 7f00552247c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:47:12.224 7f00552247c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.224 7f00552247c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:47:12.224 7f00552247c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.224 7f00552247c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.224 7f00552247c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789354032223783 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:47:12.224 7f00552247c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.225 7f00552247c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.225 7f00552247c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:47:12.225 7f00552247c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.225 7f00552247c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.226 7f00552247c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:47:12.226 7f00552247c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.226 7f00552247c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.226 7f00552247c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:47:12.226 7f00552247c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.226 7f00552247c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.226 7f00552247c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:12.226 7f00552247c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.227 7f0055224f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:47:12.227 7f0055224f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:47:12.228 7f0055224e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:47:12.228 7f00551d47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.229 7f00551d47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:47:12.229 7f00551d47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.229 7f00551d47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.229 7f00551d47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:12.229 7f00551d47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.301 7f00552247c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:47:12.309 7f00552247c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:47:12.310 7f005732f0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.311 7f005732f0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:12.311 7f005732f0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.311 7f005732f0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.311 7f005732f0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:12.311 7f005732f0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.313 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:12.315 7f00552247c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:47:12.317 7f00551d49a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.317 7f00551d49a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:12.317 7f00551d49a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.317 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:12.317 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:12.326 7f005732fd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.327 7f005732fd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:12.327 7f005732fd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.327 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:12.327 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:12.328 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:12.329 7f005732fd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.329 7f005732fd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:12.330 7f005732fd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.337 7f005732fd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.338 7f005732fd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:12.338 7f005732fd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.352 7f00552247c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:47:12.354 7f005732fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.355 7f005732fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:12.355 7f005732fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.359 7f00552247c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:47:12.361 7f005732fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.361 7f005732fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:12.361 7f005732fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.361 7f00552247c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:47:12.361 7f00552247c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:47:12.362 7f00552247c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:47:12.362 7f005732ff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:47:12.365 7f0056800220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.365 7f0056800220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:47:12.365 7f0056800220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.365 7f0056800220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.366 7f0056800220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:47:12.366 7f0056800220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.377 7f00552247c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:47:12.380 7f005732ff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:47:12.386 7f0056800540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.386 7f0056800540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:47:12.386 7f0056800540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.423 7f00552247c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:47:12.425 7f0056800ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.425 7f0056800ea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:47:12.425 7f0056800ea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:12.441 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:12.441 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:12.441 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:12.442 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:12.442 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:12.443 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:12.582 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:12.582 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:12.582 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:12.583 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:12.583 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:12.583 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:12.863 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:12.864 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:12.864 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:12.864 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:12.864 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:12.865 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:13.228 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:13.543 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:13.544 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:13.544 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:13.544 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:13.544 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:13.545 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:14.229 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:14.948 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:14.949 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:14.949 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:14.950 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:14.950 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:14.950 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:15.231 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:16.224 7f0056800ea0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:47:16.225 7f0056800ea0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:47:16.225 7f0056800ea0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:47:16.225 7f0056800ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:16.225 7f0056800ea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:47:16.226 7f0056800ea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:16.226 7f0056800ea0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:47:16.226 7f0056800ea0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:47:16.226 7f0056800ea0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:47:16.226 7f0056800ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:16.226 7f0056800ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:16.226 7f00567e3180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:47:16.227 7f00567e3180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:47:16.232 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:17.233 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:18.163 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:18.164 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:18.164 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:18.165 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:18.165 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:18.165 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:18.235 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:19.235 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:20.236 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:21.237 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:22.239 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:23.093 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:23.094 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:23.094 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:23.094 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:23.095 7f00551d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:23.095 7f00551d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:23.240 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:24.241 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:25.242 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:26.243 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:27.234 7f00551d4c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:47:27.234 7f00551d4c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:47:27.234 7f00551d4c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:27.245 7f00551d42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:27.520 7f00552247c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:47:27.520 7f005a26eb40 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:47:27.520 7f005a26eb40 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:47:27.520 7f00551d4b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:47:27.521 7f00551d4b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:47:27.537 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:47:27.537 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:47:27.537 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:47:27.537 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:47:27.537 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:47:27.537 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:47:27.537 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:47:27.538 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:47:27.538 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:47:27.538 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:47:27.538 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:47:27.538 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:47:27.538 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:47:27.538 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:47:27.538 7f005a26eb40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:47:37.572 7f676c084500 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:47:37.573 7f676c084500 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:47:37.573 7f676c084500 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:47:37.577 7f676c084500 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:47:37.578 7f67670302c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:47:37.578 7f67670302c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:47:37.578 7f67670307c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:47:37.579 7f67670307c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:47:37.579 7f67670307c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:47:37.579 7f67670307c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:47:37.579 7f67670307c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.580 7f67670307c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:47:37.580 7f67670307c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.580 7f67670307c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.580 7f67670307c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789354057579438 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:47:37.580 7f67670307c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.581 7f67670307c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.581 7f67670307c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:47:37.581 7f67670307c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.581 7f67670307c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.581 7f67670307c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:47:37.581 7f67670307c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.582 7f67670307c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.582 7f67670307c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:47:37.582 7f67670307c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.582 7f67670307c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.582 7f67670307c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:37.582 7f67670307c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.582 7f6767030e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:47:37.583 7f6767030e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:47:37.584 7f6767030c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:47:37.585 7f6766fe0540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.585 7f6766fe0540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:47:37.585 7f6766fe0540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.585 7f6766fe0540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.585 7f6766fe0540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:37.585 7f6766fe0540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.645 7f67670307c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:47:37.653 7f67670307c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:47:37.654 7f6766fc1040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.654 7f6766fc1040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:37.654 7f6766fc1040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.654 7f6766fc1040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.654 7f6766fc1040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:37.654 7f6766fc1040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.656 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:37.657 7f6766fe07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.657 7f6766fe07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:37.657 7f6766fe07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.657 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:37.658 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:37.659 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:37.659 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:37.659 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:37.661 7f6766fe07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.661 7f6766fe07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:37.662 7f6766fe07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.663 7f67670307c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:47:37.663 7f6766fe07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.664 7f6766fe07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:37.664 7f6766fe07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.672 7f6766fc1040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.673 7f6766fc1040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:37.673 7f6766fc1040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.713 7f67670307c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:47:37.714 7f6766fc1c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.714 7f6766fc1c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:37.714 7f6766fc1c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.718 7f67670307c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:47:37.718 7f6766fc1c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.718 7f6766fc1c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:37.718 7f6766fc1c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.718 7f67670307c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:47:37.719 7f67670307c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:47:37.719 7f67670307c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:47:37.719 7f6766fc1e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:47:37.720 7f6766eb20e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.720 7f6766eb20e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:47:37.720 7f6766eb20e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.720 7f6766eb20e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.720 7f6766eb20e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:47:37.720 7f6766eb20e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.723 7f67670307c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:47:37.725 7f6766fc1e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:47:37.725 7f6766eb2860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.725 7f6766eb2860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:47:37.725 7f6766eb2860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.747 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:37.748 7f67670307c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:47:37.749 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:37.749 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:37.750 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:37.750 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:37.751 7f67686c5540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.752 7f67686c5540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:47:37.752 7f67686c5540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:37.753 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:37.939 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:37.940 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:37.940 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:37.940 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:37.940 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:37.941 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:38.457 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:38.458 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:38.458 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:38.458 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:38.458 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:38.459 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:38.586 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:39.587 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:39.711 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:39.712 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:39.713 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:39.713 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:39.713 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:39.714 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:40.588 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:41.579 7f67686c5540 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:47:41.580 7f67686c5540 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:47:41.581 7f67686c5540 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:47:41.581 7f67686c5540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:41.581 7f67686c5540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:47:41.581 7f67686c5540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:41.581 7f67686c5540 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:47:41.581 7f67686c5540 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:47:41.582 7f67686c5540 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:47:41.582 7f67686c5540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:41.582 7f67686c5540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:41.582 7f67686c5a40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:47:41.583 7f67686c5a40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:47:41.590 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:42.388 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:42.389 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:42.389 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:42.389 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:42.390 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:42.390 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:42.591 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:43.593 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:44.594 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:45.595 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:46.596 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:47.581 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:47.582 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:47.582 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:47.582 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:47.582 7f6766fe0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:47.583 7f6766fe0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:47.597 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:48.598 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:49.599 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:50.601 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:51.602 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:52.603 7f6767030f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:52.604 7f6766fe0a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:47:52.604 7f6766fe0a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:47:52.605 7f6766fe0a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:52.792 7f67670307c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:47:52.792 7f676c084500 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:47:52.793 7f676c084500 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:47:52.793 7f6766fe0860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:47:52.793 7f6766fe0860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:47:52.846 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:47:52.846 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:47:52.846 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:47:52.846 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:47:52.846 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:47:52.847 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:47:52.847 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:47:52.847 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:47:52.847 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:47:52.847 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:47:52.847 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:47:52.847 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:47:52.847 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:47:52.847 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:47:52.847 7f676c084500 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:48:02.215 7fe99a5cef80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:48:02.217 7fe99a5cef80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:48:02.217 7fe99a5cef80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:48:02.222 7fe99a5cef80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:48:02.223 7fe995b3a2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:48:02.223 7fe995b3a2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:48:02.223 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:48:02.223 7fe995b3a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:48:02.224 7fe995b3a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:48:02.224 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:48:02.224 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.224 7fe995b3a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:48:02.224 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.225 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.225 7fe995b3a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789354082224260 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:48:02.225 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.225 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.226 7fe995b3a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:48:02.226 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.226 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.226 7fe995b3a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:48:02.226 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.226 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.226 7fe995b3a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:48:02.226 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.226 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.226 7fe995b3a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:02.226 7fe995b3a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.227 7fe995b3ae00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:48:02.227 7fe995b3ae00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:48:02.228 7fe995b3ac20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:48:02.229 7fe995aea540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.229 7fe995aea540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:48:02.229 7fe995aea540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.229 7fe995aea540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.229 7fe995aea540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:02.229 7fe995aea540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.294 7fe995b3a7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:48:02.303 7fe995b3a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:48:02.304 7fe997c4a040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.304 7fe997c4a040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:48:02.304 7fe997c4a040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.304 7fe997c4a040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.304 7fe997c4a040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:48:02.304 7fe997c4a040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.306 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:02.307 7fe995aea7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.307 7fe995aea7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:48:02.307 7fe995aea7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.308 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:02.308 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:02.309 7fe995b3a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:48:02.310 7fe995aea7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.310 7fe995aea7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:02.310 7fe995aea7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.310 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:02.310 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:02.311 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:02.320 7fe997c4ad60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.320 7fe997c4ad60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:02.320 7fe997c4ad60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.322 7fe997c4ad60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.323 7fe997c4ad60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:02.323 7fe997c4ad60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.348 7fe995b3a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:48:02.349 7fe997c4ac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.349 7fe997c4ac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:48:02.349 7fe997c4ac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.352 7fe995b3a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:48:02.352 7fe997c4ac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.353 7fe997c4ac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:02.353 7fe997c4ac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.353 7fe995b3a7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:48:02.353 7fe995b3a7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:48:02.353 7fe995b3a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:48:02.354 7fe997c4af40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:48:02.354 7fe997111220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.355 7fe997111220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:48:02.355 7fe997111220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.355 7fe997111220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.355 7fe997111220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:48:02.355 7fe997111220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.358 7fe995b3a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:48:02.360 7fe997c4af40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:48:02.360 7fe9971119a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.360 7fe9971119a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:48:02.360 7fe9971119a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.385 7fe995b3a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:48:02.387 7fe9971117c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.387 7fe9971117c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:48:02.387 7fe9971117c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:02.398 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:02.399 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:02.399 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:02.399 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:02.399 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:02.400 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:02.625 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:02.626 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:02.626 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:02.627 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:02.627 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:02.627 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:03.228 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:03.253 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:03.254 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:03.254 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:03.255 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:03.255 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:03.255 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:04.163 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:04.164 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:04.164 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:04.165 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:04.165 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:04.165 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:04.229 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:05.230 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:05.691 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:05.692 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:05.692 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:05.693 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:05.693 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:05.694 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:06.224 7fe9971117c0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:48:06.226 7fe9971117c0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:48:06.226 7fe9971117c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:48:06.226 7fe9971117c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:48:06.226 7fe9971117c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:48:06.226 7fe9971117c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:48:06.227 7fe9971117c0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:48:06.227 7fe9971117c0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:48:06.227 7fe9971117c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:48:06.227 7fe9971117c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:06.227 7fe9971117c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:06.228 7fe9970fa180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:48:06.229 7fe9970fa180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:48:06.231 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:07.232 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:08.233 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:09.234 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:09.706 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:09.707 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:09.707 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:09.708 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:09.708 7fe995aeaa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:09.708 7fe995aeaa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:10.235 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:11.237 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:12.238 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:13.239 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:14.241 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:15.242 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:16.243 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:17.243 7fe995b3af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:17.244 7fe995aeaa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:48:17.244 7fe995aeaa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:48:17.244 7fe995aeaa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:17.402 7fe995b3a7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:48:17.402 7fe99a5cef80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:48:17.403 7fe99a5cef80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:48:17.403 7fe995aea9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:48:17.403 7fe995aea9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:48:17.419 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:48:17.419 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:48:17.419 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:48:17.419 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:48:17.419 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:48:17.419 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:48:17.419 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:48:17.419 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:48:17.419 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:48:17.419 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:48:17.420 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:48:17.420 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:48:17.420 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:48:17.420 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:48:17.420 7fe99a5cef80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:48:26.621 7f78070a6d40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:48:26.621 7f78070a6d40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:48:26.621 7f78070a6d40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:48:26.625 7f78070a6d40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:48:26.626 7f780261e2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:48:26.626 7f780261e2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:48:26.626 7f780261e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:48:26.626 7f780261e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:48:26.627 7f780261e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:48:26.627 7f780261e7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:48:26.627 7f780261e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.627 7f780261e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:48:26.627 7f780261e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.627 7f780261e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.627 7f780261e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789354106627037 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:48:26.627 7f780261e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.632 7f780261e900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.632 7f780261e900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:48:26.632 7f780261e900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.632 7f780261e900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.632 7f780261e900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:48:26.632 7f780261e900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.633 7f780261e900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.633 7f780261e900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:48:26.633 7f780261e900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.633 7f780261e900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.633 7f780261e900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:26.633 7f780261e900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.633 7f780261ef40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:48:26.634 7f780261ef40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:48:26.635 7f780261ee00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:48:26.636 7f78025ce7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.636 7f78025ce7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:48:26.636 7f78025ce7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.636 7f78025ce7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.636 7f78025ce7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:26.636 7f78025ce7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.722 7f780261e900 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:48:26.730 7f780261e900 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:48:26.731 7f78025640e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.732 7f78025640e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:48:26.732 7f78025640e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.732 7f78025640e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.732 7f78025640e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:48:26.732 7f78025640e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.733 7f78025640e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.734 7f78025640e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:48:26.734 7f78025640e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.735 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:26.736 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:26.736 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:26.737 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:26.737 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:26.738 7f780261e900 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:48:26.738 7f78025ce9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.738 7f78025ce9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:26.738 7f78025ce9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.738 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:26.740 7f78025640e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.740 7f78025640e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:26.740 7f78025640e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.747 7f7802564d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.747 7f7802564d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:26.747 7f7802564d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.772 7f780261e900 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:48:26.773 7f7802564c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.773 7f7802564c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:48:26.774 7f7802564c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.777 7f780261e900 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:48:26.777 7f7802564c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.777 7f7802564c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:26.777 7f7802564c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.778 7f780261e900 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:48:26.778 7f780261e900 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:48:26.778 7f780261e900 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:48:26.778 7f7802564f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:48:26.779 7f78024a0220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.779 7f78024a0220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:48:26.779 7f78024a0220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.779 7f78024a0220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.779 7f78024a0220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:48:26.780 7f78024a0220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.784 7f780261e900 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:48:26.787 7f7802564f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:48:26.788 7f78024a09a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.788 7f78024a09a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:48:26.788 7f78024a09a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.817 7f780261e900 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:48:26.819 7f7803cb4180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.819 7f7803cb4180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:48:26.819 7f7803cb4180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.828 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:26.829 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:26.829 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:26.830 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:26.830 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:26.830 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:27.068 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:27.069 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:27.070 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:27.071 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:27.072 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:27.072 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:27.440 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:27.441 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:27.441 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:27.442 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:27.442 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:27.442 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:27.634 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:28.473 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:28.474 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:28.474 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:28.475 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:28.475 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:28.475 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:28.636 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:29.637 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:30.411 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:30.411 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:30.412 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:30.412 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:30.412 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:30.413 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:30.627 7f7803cb4680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:48:30.628 7f7803cb4680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:48:30.629 7f7803cb4680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:48:30.629 7f7803cb4680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:48:30.629 7f7803cb4680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:48:30.629 7f7803cb4680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:48:30.629 7f7803cb4680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:48:30.629 7f7803cb4680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:48:30.629 7f7803cb4680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:48:30.629 7f7803cb4680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:30.630 7f7803cb4680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:30.630 7f7803cb4900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:48:30.631 7f7803cb4900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:48:30.637 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:31.639 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:32.640 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:33.532 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:33.533 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:33.533 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:33.534 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:33.534 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:33.534 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:33.641 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:34.642 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:35.643 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:36.644 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:37.645 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:38.646 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:39.647 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:40.006 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:40.007 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:40.007 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:40.008 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:40.008 7f78025cec20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:40.008 7f78025cec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:40.649 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:41.653 7f78025ce2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:41.713 7f78025cec20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:48:41.713 7f78025cec20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:48:41.714 7f78025cec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:41.915 7f780261e900 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:48:41.916 7f78070a6d40 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:48:41.916 7f78070a6d40 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:48:41.916 7f78025ce860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:48:41.916 7f78025ce860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:48:41.941 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:48:41.942 7f78070a6d40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:48:51.086 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:48:51.087 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:48:51.088 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:48:51.092 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:48:51.092 7f5dcff3f2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:48:51.093 7f5dcff3f2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:48:51.093 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:48:51.093 7f5dcff3f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:48:51.094 7f5dcff3f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:48:51.094 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:48:51.094 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.094 7f5dcff3f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:48:51.094 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.094 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.094 7f5dcff3f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789354131094225 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:48:51.095 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.095 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.095 7f5dcff3f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:48:51.096 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.096 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.096 7f5dcff3f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:48:51.096 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.096 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.096 7f5dcff3f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:48:51.096 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.096 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.096 7f5dcff3f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:51.096 7f5dcff3f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.097 7f5dcff3fe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:48:51.097 7f5dcff3fe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:48:51.098 7f5dcff3fc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:48:51.099 7f5dcfeef540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.099 7f5dcfeef540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:48:51.099 7f5dcfeef540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.099 7f5dcfeef540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.099 7f5dcfeef540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:51.099 7f5dcfeef540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.161 7f5dcff3f7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:48:51.172 7f5dcff3f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:48:51.173 7f5dcfea90e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.173 7f5dcfea90e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:48:51.173 7f5dcfea90e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.173 7f5dcfea90e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.174 7f5dcfea90e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:48:51.174 7f5dcfea90e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.177 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:51.179 7f5dcff3f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:48:51.186 7f5dcfeef7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.186 7f5dcfeef7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:51.186 7f5dcfeef7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.188 7f5dcfea9e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.188 7f5dcfea9e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:51.188 7f5dcfea9e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.189 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:51.189 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:51.189 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:51.189 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:51.190 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:51.192 7f5dcfea9e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.193 7f5dcfea9e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:51.193 7f5dcfea9e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.195 7f5dcfea9e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.195 7f5dcfea9e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:51.195 7f5dcfea9e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.227 7f5dcff3f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:48:51.228 7f5dcfeef7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.228 7f5dcfeef7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:48:51.228 7f5dcfeef7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.232 7f5dcff3f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:48:51.232 7f5dcfeef7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.232 7f5dcfeef7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:48:51.232 7f5dcfeef7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.233 7f5dcff3f7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:48:51.233 7f5dcff3f7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:48:51.233 7f5dcff3f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:48:51.233 7f5dcfeef7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:48:51.234 7f5dd151e220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.235 7f5dd151e220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:48:51.235 7f5dd151e220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.235 7f5dd151e220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.235 7f5dd151e220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:48:51.236 7f5dd151e220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.241 7f5dcff3f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:48:51.244 7f5dcfeef7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:48:51.244 7f5dd151e860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.245 7f5dd151e860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:48:51.245 7f5dd151e860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.267 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:51.268 7f5dcff3f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:48:51.269 7f5dd14f22c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.270 7f5dd14f22c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:48:51.270 7f5dd14f22c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:51.270 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:51.270 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:51.272 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:51.272 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:51.273 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:51.433 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:51.438 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:51.438 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:51.439 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:51.439 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:51.439 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:51.751 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:51.751 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:51.752 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:51.752 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:51.752 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:51.754 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:52.098 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:52.516 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:52.517 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:52.517 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:52.517 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:52.517 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:52.518 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:53.099 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:54.101 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:54.567 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:54.568 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:54.568 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:54.568 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:54.569 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:54.569 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:55.094 7f5dd14f27c0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:48:55.095 7f5dd14f27c0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:48:55.096 7f5dd14f27c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:48:55.096 7f5dd14f27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:48:55.096 7f5dd14f27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:48:55.096 7f5dd14f27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:48:55.096 7f5dd14f27c0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:48:55.096 7f5dd14f27c0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:48:55.096 7f5dd14f27c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:48:55.096 7f5dd14f27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:55.097 7f5dd14f27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:55.097 7f5dd14f2a40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:48:55.098 7f5dd14f2a40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:48:55.102 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:56.103 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:57.104 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:58.105 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:59.107 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:48:59.258 7f5dd06e0c20 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:48:59.995 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:48:59.996 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:59.996 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:48:59.996 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:48:59.996 7f5dcfeefc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:48:59.997 7f5dcfeefc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:00.108 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:49:01.110 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:49:02.111 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:49:03.113 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:49:04.114 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:49:05.119 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:49:06.120 7f5dcff3ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:49:06.132 7f5dcfeefc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:49:06.132 7f5dcfeefc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:49:06.132 7f5dd151e540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:49:06.321 7f5dcff3f7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:49:06.321 7f5dd49c6e00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:49:06.321 7f5dd49c6e00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:49:06.321 7f5dcfeefc20 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:49:06.322 7f5dcfeefc20 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:49:06.340 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:49:06.340 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:49:06.340 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:49:06.340 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:49:06.340 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:49:06.340 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:49:06.340 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:49:06.340 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:49:06.340 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:49:06.340 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:49:06.340 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:49:06.340 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:49:06.340 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:49:06.341 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:49:06.341 7f5dd49c6e00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:49:16.730 7f306e2f4d40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:49:16.731 7f306e2f4d40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:49:16.731 7f306e2f4d40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:49:16.738 7f306e2f4d40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:49:16.738 7f306929f2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:49:16.738 7f306929f2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:49:16.739 7f306929f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:49:16.739 7f306929f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:49:16.739 7f306929f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:49:16.739 7f306929f7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:49:16.740 7f306929f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.740 7f306929f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:49:16.740 7f306929f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.740 7f306929f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.740 7f306929f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789354156739706 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:49:16.740 7f306929f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.741 7f306929f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.741 7f306929f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:49:16.741 7f306929f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.741 7f306929f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.741 7f306929f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:49:16.741 7f306929f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.741 7f306929f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.741 7f306929f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:49:16.742 7f306929f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.742 7f306929f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.742 7f306929f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:49:16.742 7f306929f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.742 7f306929fe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:49:16.742 7f306929fe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:49:16.743 7f306929fc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:49:16.744 7f306924f540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.744 7f306924f540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:49:16.744 7f306924f540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.744 7f306924f540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.744 7f306924f540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:49:16.744 7f306924f540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.804 7f306929f7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:49:16.812 7f306929f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:49:16.813 7f30691aa040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.813 7f30691aa040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:49:16.813 7f30691aa040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.813 7f30691aa040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.813 7f30691aa040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:49:16.813 7f30691aa040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.815 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:49:16.816 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:16.817 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:49:16.818 7f306924f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.819 7f306924f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:49:16.819 7f306924f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.819 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:16.819 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:49:16.819 7f306929f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:49:16.820 7f306924f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.820 7f306924f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:49:16.820 7f306924f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.821 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:16.822 7f306924f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.822 7f306924f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:49:16.822 7f306924f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.830 7f30691aac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.830 7f30691aac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:49:16.830 7f30691aac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.909 7f306929f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:49:16.910 7f30691aaae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.910 7f30691aaae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:49:16.910 7f30691aaae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.914 7f306929f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:49:16.915 7f30691aaae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.915 7f30691aaae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:49:16.915 7f30691aaae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.916 7f306929f7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:49:16.916 7f306929f7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:49:16.916 7f306929f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:49:16.916 7f30691aae00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:49:16.917 7f30691220e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.917 7f30691220e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:49:16.917 7f30691220e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.917 7f30691220e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.917 7f30691220e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:49:16.917 7f30691220e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.922 7f306929f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:49:16.924 7f30691aae00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:49:16.925 7f3069122860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.926 7f3069122860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:49:16.927 7f3069122860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.968 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:49:16.969 7f306929f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:49:16.971 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:16.971 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:49:16.976 7f306a862540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.976 7f306a862540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:49:16.976 7f306a862540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:16.979 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:16.979 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:49:16.983 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:17.217 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:49:17.218 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:17.218 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:49:17.218 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:17.218 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:49:17.219 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:17.672 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:49:17.673 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:17.673 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:49:17.674 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:17.674 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:49:17.675 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:17.743 7f306929ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:49:18.318 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:49:18.337 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:18.337 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:49:18.338 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:18.338 7f306924fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:49:18.338 7f306924fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:49:18.744 7f306929ff40 [contrail-alarm-gen] [INFO]: UVE Done