09/01/2025 02:12:50.495 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:50.495 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:50.495 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:50.495 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:50.495 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:50.495 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:50.495 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:50.495 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:50.495 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:50.496 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:50.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:51.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:52.377 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:12:52.377 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:12:52.377 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:12:52.377 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:12:52.377 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:12:52.377 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:12:52.377 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:12:52.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:53.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:54.495 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:54.496 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:54.496 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:54.496 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:54.497 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:54.497 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:54.497 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:54.497 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:54.497 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:54.497 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:54.497 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:54.497 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:54.497 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:54.497 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:54.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:55.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:56.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:57.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:58.498 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:58.499 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:58.499 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:58.499 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:58.499 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:58.499 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:58.499 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:58.499 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:58.499 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:58.499 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:58.499 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:58.500 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:58.500 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:58.500 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:58.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:59.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:00.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:01.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:02.499 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:02.500 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:02.500 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:02.500 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:02.500 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:02.501 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:02.501 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:02.501 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:02.501 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:02.501 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:02.501 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:02.501 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:02.501 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:02.501 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:02.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:03.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:04.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:05.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:06.501 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:06.502 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:06.502 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:06.502 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:06.502 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:06.502 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:06.503 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:06.503 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:06.503 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:06.503 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:06.503 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:06.503 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:06.503 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:06.503 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:06.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:07.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:08.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:09.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:10.503 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:10.504 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:10.504 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:10.504 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:10.504 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:10.504 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:10.504 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:10.504 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:10.504 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:10.505 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:10.505 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:10.505 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:10.505 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:10.505 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:10.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:11.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:12.851 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:13.852 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:14.505 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:14.506 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:14.506 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:14.506 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:14.506 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:14.506 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:14.506 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:14.507 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:14.507 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:14.507 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:14.507 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:14.507 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:14.507 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:14.507 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:14.853 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:15.854 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:16.856 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:18.507 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:18.508 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:18.508 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:18.508 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:18.509 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:18.509 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:18.509 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:18.509 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:18.509 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:18.509 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:18.509 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:18.509 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:18.509 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:18.509 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:18.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:19.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:20.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:21.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:22.509 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:22.510 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:22.510 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:22.510 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:22.510 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:22.510 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:22.510 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:22.511 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:22.511 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:22.511 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:22.511 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:22.511 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:22.511 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:22.511 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:22.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:23.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:24.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:25.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:26.510 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:26.511 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:26.512 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:26.512 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:26.512 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:26.512 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:26.512 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:26.512 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:26.512 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:26.512 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:26.512 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:26.512 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:26.512 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:26.512 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:26.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:27.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:28.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:29.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:30.512 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:30.513 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:30.513 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:30.513 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:30.513 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:30.513 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:30.514 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:30.514 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:30.514 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:30.514 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:30.514 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:30.514 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:30.514 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:30.514 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:30.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:31.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:32.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:33.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:34.514 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:34.515 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:34.515 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:34.515 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:34.515 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:34.516 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:34.516 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:34.516 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:34.516 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:34.516 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:34.516 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:34.516 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:34.516 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:34.516 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:34.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:35.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:36.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:37.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:38.517 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:38.518 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:38.518 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:38.518 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:38.518 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:38.518 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:38.518 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:38.518 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:38.518 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:38.519 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:38.519 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:38.519 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:38.519 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:38.519 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:38.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:39.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:40.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:41.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:42.519 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:42.519 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:42.520 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:42.520 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:42.520 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:42.520 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:42.520 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:42.520 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:42.520 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:42.520 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:42.520 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:42.520 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:42.520 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:42.520 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:42.848 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:43.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:44.851 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:45.853 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:46.520 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:46.521 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:46.521 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:46.521 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:46.521 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:46.522 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:46.522 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:46.522 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:46.522 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:46.522 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:46.522 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:46.522 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:46.522 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:46.522 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:46.854 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:48.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:49.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:50.522 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:50.523 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:50.523 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:50.523 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:50.524 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:50.524 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:50.524 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:50.524 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:50.524 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:50.524 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:50.524 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:50.524 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:50.524 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:50.524 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:50.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:51.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:52.401 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:13:52.402 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:13:52.402 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:13:52.402 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:13:52.402 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:13:52.402 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:13:52.402 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:13:52.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:53.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:54.525 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:54.526 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:54.526 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:54.526 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:54.526 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:54.526 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:54.526 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:54.527 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:54.527 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:54.527 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:54.527 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:54.527 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:54.527 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:54.527 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:54.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:55.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:56.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:57.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:58.527 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:58.528 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:58.528 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:58.528 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:58.528 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:58.529 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:58.529 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:58.529 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:58.529 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:58.529 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:58.529 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:58.529 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:58.529 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:13:58.530 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:58.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:13:59.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:00.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:01.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:02.529 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:02.530 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:02.530 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:02.530 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:02.530 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:02.530 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:02.530 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:02.531 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:02.531 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:02.531 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:02.531 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:02.531 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:02.531 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:02.531 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:02.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:03.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:04.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:05.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:06.530 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:06.532 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:06.532 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:06.532 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:06.532 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:06.532 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:06.532 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:06.532 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:06.532 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:06.533 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:06.533 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:06.533 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:06.533 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:06.533 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:06.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:07.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:08.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:09.848 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:10.533 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:10.534 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:10.534 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:10.534 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:10.534 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:10.534 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:10.534 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:10.534 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:10.534 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:10.534 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:10.534 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:10.535 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:10.535 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:10.535 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:10.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:11.850 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:12.852 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:13.855 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:14.534 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:14.536 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:14.536 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:14.536 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:14.536 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:14.536 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:14.536 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:14.536 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:14.537 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:14.537 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:14.537 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:14.537 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:14.537 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:14.537 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:14.856 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:15.857 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:16.858 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:18.537 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:18.538 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:18.539 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:18.539 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:18.539 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:18.539 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:18.539 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:18.539 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:18.539 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:18.539 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:18.539 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:18.539 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:18.539 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:18.539 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:18.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:19.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:20.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:21.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:22.539 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:22.540 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:22.541 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:22.541 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:22.541 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:22.541 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:22.541 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:22.541 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:22.541 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:22.541 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:22.541 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:22.541 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:22.541 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:22.541 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:22.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:23.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:24.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:25.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:26.541 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:26.542 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:26.542 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:26.542 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:26.542 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:26.542 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:26.543 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:26.543 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:26.543 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:26.543 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:26.543 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:26.543 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:26.543 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:26.544 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:26.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:27.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:28.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:29.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:30.543 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:30.544 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:30.544 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:30.544 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:30.544 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:30.545 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:30.545 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:30.545 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:30.545 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:30.545 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:30.545 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:30.545 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:30.545 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:30.545 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:30.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:31.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:32.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:33.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:34.545 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:34.546 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:34.546 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:34.547 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:34.547 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:34.547 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:34.547 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:34.547 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:34.547 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:34.547 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:34.547 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:34.547 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:34.547 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:34.547 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:34.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:35.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:36.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:37.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:38.547 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:38.548 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:38.549 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:38.549 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:38.549 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:38.549 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:38.549 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:38.549 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:38.549 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:38.549 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:38.549 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:38.549 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:38.549 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:38.550 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:38.945 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:39.947 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:40.948 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:41.949 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:42.550 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:42.551 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:42.551 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:42.551 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:42.551 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:42.551 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:42.551 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:42.551 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:42.552 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:42.552 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:42.552 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:42.552 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:42.552 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:42.552 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:42.951 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:43.953 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:44.954 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:45.956 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:46.551 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:46.553 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:46.553 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:46.553 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:46.553 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:46.553 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:46.553 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:46.554 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:46.554 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:46.554 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:46.554 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:46.554 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:46.554 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:46.554 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:46.958 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:48.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:49.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:50.554 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:50.557 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:50.558 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:50.558 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:50.558 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:50.558 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:50.558 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:50.558 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:50.559 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:50.559 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:50.559 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:50.559 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:50.559 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:50.559 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:50.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:51.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:52.030 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:14:52.032 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:14:52.429 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:14:52.429 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:14:52.429 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:14:52.429 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:14:52.429 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:14:52.430 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:14:52.430 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:14:52.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:53.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:54.558 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:54.559 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:54.559 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:54.559 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:54.560 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:54.560 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:54.560 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:54.560 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:54.560 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:54.561 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:54.561 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:54.561 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:54.561 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:54.561 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:54.577 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:14:54.580 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:14:54.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:55.871 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:56.873 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:57.874 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:58.243 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:14:58.246 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:14:58.560 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:58.561 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:58.561 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:58.561 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:58.562 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:58.562 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:58.562 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:58.562 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:58.562 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:58.562 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:58.562 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:58.562 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:58.562 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:14:58.562 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:58.876 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:14:59.877 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:00.878 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:01.880 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:02.561 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:02.562 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:02.563 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:02.563 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:02.563 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:02.563 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:02.563 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:02.563 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:02.563 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:02.563 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:02.563 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:02.563 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:02.563 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:02.563 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:02.881 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:03.882 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:04.884 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:05.884 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:06.563 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:06.565 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:06.565 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:06.565 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:06.566 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:06.566 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:06.566 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:06.566 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:06.566 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:06.567 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:06.567 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:06.567 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:06.567 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:06.567 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:06.886 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:07.888 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:08.889 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:09.891 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:10.565 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:10.566 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:10.566 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:10.566 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:10.566 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:10.566 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:10.566 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:10.566 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:10.567 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:10.567 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:10.567 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:10.567 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:10.567 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:10.567 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:10.892 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:11.894 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:12.895 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:13.897 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:14.567 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:14.569 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:14.569 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:14.569 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:14.569 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:14.569 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:14.569 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:14.569 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:14.569 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:14.570 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:14.570 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:14.570 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:14.570 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:14.570 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:14.898 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:15.900 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:16.902 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:18.570 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:18.571 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:18.572 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:18.572 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:18.572 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:18.572 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:18.572 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:18.572 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:18.572 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:18.572 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:18.572 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:18.572 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:18.572 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:18.573 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:18.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:19.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:20.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:21.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:22.572 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:22.573 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:22.573 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:22.573 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:22.573 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:22.573 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:22.574 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:22.574 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:22.574 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:22.574 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:22.574 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:22.574 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:22.574 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:22.574 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:22.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:23.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:24.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:25.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:26.574 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:26.575 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:26.575 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:26.575 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:26.576 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:26.576 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:26.576 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:26.576 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:26.576 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:26.576 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:26.576 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:26.576 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:26.576 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:26.576 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:26.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:27.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:28.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:29.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:30.577 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:30.578 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:30.578 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:30.578 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:30.578 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:30.578 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:30.578 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:30.579 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:30.579 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:30.579 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:30.579 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:30.579 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:30.579 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:30.579 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:30.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:31.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:32.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:33.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:34.579 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:34.580 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:34.581 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:34.581 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:34.581 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:34.581 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:34.581 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:34.581 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:34.581 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:34.581 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:34.581 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:34.581 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:34.581 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:34.581 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:34.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:35.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:36.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:37.848 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:38.581 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:38.582 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:38.583 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:38.583 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:38.583 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:38.583 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:38.583 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:38.583 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:38.583 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:38.583 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:38.583 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:38.583 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:38.583 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:38.583 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:38.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:39.850 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:40.852 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:41.853 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:42.584 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:42.585 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:42.585 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:42.585 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:42.585 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:42.585 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:42.585 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:42.585 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:42.585 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:42.585 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:42.586 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:42.586 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:42.586 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:42.586 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:42.854 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:43.856 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:44.858 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:45.858 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:46.585 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:46.586 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:46.586 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:46.587 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:46.587 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:46.587 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:46.587 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:46.587 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:46.587 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:46.587 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:46.587 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:46.587 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:46.587 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:46.587 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:46.860 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:48.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:49.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:50.588 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:50.589 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:50.589 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:50.589 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:50.589 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:50.589 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:50.589 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:50.589 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:50.589 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:50.590 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:50.590 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:50.590 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:50.590 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:50.590 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:50.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:51.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:52.455 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:15:52.455 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:15:52.456 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:15:52.456 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:15:52.456 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:15:52.456 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:15:52.456 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:15:52.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:53.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:54.590 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:54.591 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:54.591 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:54.591 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:54.592 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:54.592 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:54.592 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:54.592 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:54.592 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:54.592 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:54.592 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:54.592 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:54.592 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:54.592 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:54.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:55.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:56.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:57.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:58.592 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:58.593 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:58.593 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:58.593 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:58.593 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:58.593 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:58.593 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:58.594 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:58.594 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:58.594 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:58.594 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:58.594 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:58.594 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:15:58.594 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:58.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:15:59.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:00.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:01.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:02.594 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:02.595 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:02.596 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:02.596 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:02.596 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:02.596 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:02.596 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:02.596 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:02.596 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:02.596 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:02.596 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:02.596 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:02.597 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:02.597 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:02.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:03.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:04.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:05.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:06.597 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:06.597 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:06.598 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:06.598 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:06.598 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:06.598 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:06.598 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:06.598 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:06.598 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:06.598 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:06.598 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:06.598 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:06.598 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:06.599 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:06.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:07.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:08.851 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:09.852 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:10.598 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:10.599 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:10.599 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:10.600 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:10.600 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:10.600 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:10.600 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:10.600 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:10.600 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:10.600 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:10.600 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:10.600 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:10.600 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:10.600 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:10.853 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:11.854 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:12.855 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:13.857 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:14.601 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:14.602 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:14.602 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:14.602 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:14.602 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:14.602 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:14.602 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:14.602 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:14.603 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:14.603 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:14.603 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:14.603 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:14.603 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:14.603 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:14.858 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:15.860 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:16.862 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:18.603 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:18.605 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:18.605 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:18.605 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:18.605 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:18.605 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:18.605 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:18.605 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:18.605 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:18.605 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:18.606 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:18.606 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:18.606 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:18.606 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:18.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:19.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:20.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:21.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:22.606 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:22.607 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:22.607 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:22.607 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:22.607 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:22.607 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:22.607 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:22.608 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:22.608 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:22.608 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:22.608 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:22.608 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:22.608 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:22.608 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:22.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:23.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:24.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:25.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:26.608 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:26.609 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:26.609 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:26.609 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:26.609 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:26.609 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:26.609 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:26.609 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:26.609 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:26.609 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:26.609 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:26.610 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:26.610 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:26.610 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:26.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:27.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:28.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:29.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:30.610 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:30.612 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:30.612 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:30.612 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:30.612 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:30.612 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:30.612 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:30.613 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:30.613 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:30.613 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:30.613 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:30.613 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:30.613 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:30.613 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:30.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:31.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:32.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:33.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:34.613 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:34.614 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:34.614 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:34.614 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:34.614 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:34.614 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:34.614 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:34.614 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:34.615 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:34.615 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:34.615 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:34.615 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:34.615 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:34.615 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:34.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:35.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:36.848 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:37.116 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:16:37.119 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:16:37.850 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:38.614 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:38.616 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:38.616 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:38.616 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:38.616 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:38.616 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:38.616 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:38.617 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:38.617 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:38.617 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:38.617 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:38.617 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:38.617 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:38.617 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:38.851 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:39.852 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:40.854 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:41.856 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:42.617 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:42.618 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:42.618 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:42.618 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:42.618 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:42.618 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:42.618 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:42.618 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:42.619 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:42.619 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:42.619 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:42.619 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:42.619 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:42.619 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:42.857 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:43.859 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:44.860 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:45.862 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:46.618 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:46.619 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:46.620 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:46.620 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:46.620 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:46.620 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:46.620 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:46.620 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:46.620 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:46.620 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:46.620 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:46.620 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:46.621 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:46.621 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:46.863 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:48.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:49.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:50.620 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:50.621 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:50.621 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:50.622 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:50.622 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:50.622 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:50.622 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:50.622 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:50.622 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:50.622 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:50.622 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:50.622 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:50.622 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:50.622 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:50.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:51.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:52.481 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:16:52.481 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:16:52.481 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:16:52.481 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:16:52.482 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:16:52.482 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:16:52.482 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:16:52.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:53.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:54.623 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:54.624 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:54.624 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:54.624 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:54.624 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:54.624 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:54.624 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:54.624 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:54.624 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:54.624 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:54.625 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:54.625 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:54.625 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:54.625 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:54.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:55.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:56.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:57.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:58.625 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:58.626 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:58.626 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:58.626 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:58.626 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:58.626 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:58.626 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:58.626 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:58.627 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:58.627 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:58.627 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:58.627 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:58.627 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:16:58.627 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:58.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:16:59.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:00.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:01.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:02.626 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:02.628 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:02.628 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:02.628 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:02.628 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:02.628 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:02.628 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:02.629 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:02.629 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:02.629 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:02.629 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:02.629 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:02.629 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:02.629 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:02.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:03.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:04.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:05.848 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:06.629 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:06.630 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:06.630 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:06.630 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:06.630 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:06.630 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:06.630 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:06.630 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:06.630 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:06.631 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:06.631 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:06.631 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:06.631 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:06.631 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:06.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:07.851 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:08.853 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:09.854 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:10.631 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:10.632 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:10.632 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:10.632 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:10.632 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:10.632 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:10.632 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:10.633 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:10.633 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:10.633 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:10.633 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:10.633 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:10.633 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:10.633 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:10.856 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:11.856 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:12.858 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:13.859 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:14.633 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:14.634 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:14.634 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:14.634 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:14.634 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:14.634 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:14.634 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:14.635 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:14.635 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:14.635 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:14.635 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:14.635 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:14.635 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:14.635 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:14.861 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:15.862 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:16.871 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:18.635 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:18.636 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:18.637 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:18.637 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:18.637 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:18.637 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:18.637 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:18.637 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:18.637 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:18.637 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:18.637 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:18.637 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:18.637 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:18.637 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:18.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:19.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:20.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:21.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:22.637 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:22.638 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:22.638 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:22.638 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:22.638 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:22.638 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:22.638 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:22.639 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:22.639 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:22.639 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:22.639 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:22.639 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:22.639 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:22.639 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:22.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:23.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:24.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:25.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:26.639 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:26.640 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:26.640 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:26.640 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:26.640 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:26.640 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:26.640 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:26.640 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:26.640 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:26.640 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:26.641 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:26.641 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:26.641 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:26.641 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:26.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:27.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:28.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:29.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:30.641 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:30.642 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:30.642 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:30.642 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:30.643 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:30.643 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:30.643 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:30.643 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:30.643 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:30.643 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:30.643 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:30.643 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:30.643 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:30.644 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:30.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:31.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:32.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:33.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:34.644 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:34.645 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:34.645 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:34.645 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:34.645 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:34.645 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:34.645 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:34.646 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:34.646 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:34.646 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:34.646 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:34.646 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:34.646 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:34.646 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:34.848 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:35.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:36.850 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:37.852 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:38.646 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:38.647 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:38.647 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:38.647 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:38.647 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:38.647 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:38.647 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:38.647 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:38.647 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:38.648 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:38.648 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:38.648 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:38.648 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:38.648 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:38.853 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:39.855 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:40.857 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:41.858 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:42.647 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:42.648 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:42.648 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:42.648 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:42.648 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:42.649 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:42.649 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:42.649 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:42.649 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:42.649 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:42.649 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:42.649 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:42.649 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:42.649 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:42.860 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:43.861 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:44.862 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:45.863 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:46.649 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:46.651 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:46.651 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:46.651 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:46.651 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:46.651 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:46.651 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:46.651 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:46.652 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:46.652 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:46.652 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:46.652 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:46.652 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:46.652 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:46.864 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:48.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:49.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:50.652 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:50.653 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:50.653 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:50.653 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:50.654 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:50.654 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:50.654 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:50.654 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:50.654 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:50.654 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:50.654 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:50.654 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:50.655 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:50.655 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:50.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:51.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:52.503 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:17:52.504 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:17:52.504 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:17:52.504 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:17:52.504 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:17:52.504 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:17:52.504 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:17:52.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:53.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:54.655 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:54.656 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:54.656 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:54.656 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:54.656 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:54.656 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:54.656 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:54.656 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:54.656 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:54.656 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:54.657 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:54.657 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:54.657 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:54.657 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:54.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:55.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:56.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:57.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:58.657 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:58.658 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:58.658 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:58.658 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:58.658 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:58.658 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:58.658 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:58.659 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:58.659 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:58.659 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:58.659 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:58.659 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:58.659 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:17:58.659 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:58.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:17:59.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:00.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:01.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:02.659 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:02.660 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:02.660 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:02.661 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:02.661 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:02.661 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:02.661 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:02.661 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:02.661 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:02.661 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:02.661 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:02.661 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:02.662 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:02.662 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:02.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:03.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:04.848 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:05.850 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:06.661 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:06.662 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:06.662 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:06.663 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:06.663 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:06.663 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:06.663 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:06.663 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:06.663 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:06.663 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:06.663 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:06.663 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:06.663 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:06.663 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:06.851 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:07.853 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:08.854 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:09.856 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:10.664 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:10.665 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:10.665 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:10.665 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:10.665 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:10.665 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:10.665 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:10.666 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:10.666 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:10.666 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:10.666 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:10.666 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:10.666 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:10.666 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:10.857 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:11.859 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:12.860 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:13.862 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:14.666 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:14.667 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:14.667 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:14.667 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:14.667 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:14.667 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:14.667 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:14.668 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:14.668 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:14.668 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:14.668 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:14.668 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:14.668 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:14.668 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:14.863 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:15.863 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:16.866 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:18.668 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:18.669 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:18.670 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:18.670 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:18.670 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:18.670 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:18.670 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:18.670 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:18.670 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:18.670 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:18.670 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:18.670 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:18.670 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:18.670 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:18.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:19.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:20.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:21.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:22.671 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:22.672 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:22.672 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:22.672 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:22.672 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:22.672 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:22.672 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:22.673 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:22.673 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:22.673 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:22.673 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:22.673 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:22.673 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:22.673 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:22.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:23.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:24.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:25.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:26.674 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:26.675 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:26.675 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:26.675 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:26.675 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:26.675 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:26.675 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:26.675 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:26.675 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:26.675 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:26.675 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:26.675 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:26.676 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:26.676 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:26.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:27.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:28.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:29.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:30.675 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:30.677 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:30.677 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:30.677 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:30.677 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:30.677 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:30.677 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:30.677 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:30.677 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:30.677 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:30.677 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:30.677 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:30.678 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:30.678 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:30.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:31.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:32.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:33.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:34.677 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:34.678 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:34.678 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:34.678 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:34.678 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:34.678 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:34.678 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:34.679 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:34.679 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:34.679 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:34.679 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:34.679 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:34.679 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:34.679 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:34.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:35.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:36.848 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:37.850 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:38.680 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:38.681 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:38.681 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:38.681 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:38.681 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:38.681 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:38.681 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:38.682 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:38.682 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:38.682 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:38.682 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:38.682 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:38.682 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:38.682 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:38.851 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:39.852 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:40.854 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:41.855 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:42.682 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:42.684 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:42.684 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:42.684 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:42.684 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:42.684 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:42.684 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:42.684 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:42.684 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:42.684 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:42.685 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:42.685 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:42.685 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:42.685 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:42.856 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:43.857 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:44.859 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:45.860 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:46.684 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:46.686 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:46.686 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:46.686 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:46.686 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:46.686 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:46.686 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:46.687 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:46.687 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:46.687 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:46.687 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:46.687 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:46.687 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:46.687 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:46.862 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:48.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:49.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:50.687 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:50.688 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:50.688 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:50.688 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:50.688 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:50.688 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:50.688 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:50.689 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:50.689 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:50.689 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:50.689 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:50.689 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:50.689 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:50.689 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:50.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:51.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:52.530 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:18:52.530 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:18:52.530 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:18:52.530 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:18:52.530 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:18:52.530 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:18:52.530 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:18:52.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:53.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:54.689 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:54.690 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:54.691 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:54.691 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:54.691 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:54.691 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:54.691 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:54.691 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:54.692 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:54.692 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:54.692 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:54.692 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:54.692 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:18:54.692 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:54.766 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:18:54.769 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:18:54.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:18:55.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process.