05/01/2025 02:07:23.476 7f7bd9147ac8 [contrail-analytics-api] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 05/01/2025 02:07:23.476 7f7bd9147ac8 [contrail-analytics-api] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 05/01/2025 02:07:23.476 7f7bd9147ac8 [contrail-analytics-api] [INFO]: SANDESH: Flow Logging: [None] -> [False] 05/01/2025 02:07:23.477 7f7bd9147ac8 [contrail-analytics-api] [DEBUG]: Add uve in the [ObjectCollectorInfo:NodeStatusUVE] map 05/01/2025 02:07:23.500 7f7bd9147ac8 [contrail-analytics-api] [DEBUG]: Add uve in the [ObjectCollectorInfo:AnalyticsApiInfoUVE] map 05/01/2025 02:07:23.501 7f7bd14fc438 [contrail-analytics-api] [ERROR]: Cannot write http_port 8090 to /tmp/contrail-analytics-api.0.http_port 05/01/2025 02:07:23.501 7f7bd14fc438 [contrail-analytics-api] [ERROR]: Starting Introspect on HTTP Port 8090 05/01/2025 02:07:23.501 7f7bd1502d68 [contrail-analytics-api] [INFO]: Processing event[EvStart] in state[none] 05/01/2025 02:07:23.501 7f7bd1502d68 [contrail-analytics-api] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 05/01/2025 02:07:23.501 7f7bd1502d68 [contrail-analytics-api] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 05/01/2025 02:07:23.501 7f7bd1502d68 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.502 7f7bd1502d68 [contrail-analytics-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3254-1. process_status = [ << module_id = contrail-analytics-api instance_id = 0 state = Functional connection_infos = [ << type = UvePartitions name = UVE-Aggregation server_addrs = [ 10.0.0.241:6379, ] status = Up description = Not using Kafka for UVE Aggregation >>, ] description = >>, ] >> 05/01/2025 02:07:23.502 7f7bd1502d68 [contrail-analytics-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.502 7f7bd1502d68 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.502 7f7bd1502d68 [contrail-analytics-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3254-1. process_status = [ << module_id = contrail-analytics-api instance_id = 0 state = Non-Functional connection_infos = [ << type = UvePartitions name = UVE-Aggregation server_addrs = [ 10.0.0.241:6379, ] status = Up description = Not using Kafka for UVE Aggregation >>, << type = Redis-UVE name = 10.0.0.241:6379 server_addrs = [ 10.0.0.241:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Redis-UVE:10.0.0.241:6379[Redis Instance initialized] connection down >>, ] >> 05/01/2025 02:07:23.502 7f7bd1502d68 [contrail-analytics-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.502 7f7bd1502d68 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.502 7f7bd1502d68 [contrail-analytics-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3254-1. process_status = [ << module_id = contrail-analytics-api instance_id = 0 state = Non-Functional connection_infos = [ << type = UvePartitions name = UVE-Aggregation server_addrs = [ 10.0.0.241:6379, ] status = Up description = Not using Kafka for UVE Aggregation >>, << type = Redis-UVE name = 10.0.0.241:6379 server_addrs = [ 10.0.0.241:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = OpServer server_addrs = [ 10.0.0.241:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Redis-UVE:10.0.0.241:6379[Redis Instance initialized], Zookeeper:OpServer[Connection to Zookeeper initialized] connection down >>, ] >> 05/01/2025 02:07:23.502 7f7bd1502d68 [contrail-analytics-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.502 7f7bd1502d68 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.502 7f7bd1502d68 [contrail-analytics-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] AnalyticsApiInfoUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3254-1. analytics_node_ip = { rest_api_ip : 10.0.0.241host_ip : 10.0.0.241 } >> 05/01/2025 02:07:23.502 7f7bd1502d68 [contrail-analytics-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.502 7f7bd1502d68 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.503 7f7bd1502d68 [contrail-analytics-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3254-1. process_status = [ << module_id = contrail-analytics-api instance_id = 0 state = Non-Functional connection_infos = [ << type = UvePartitions name = UVE-Aggregation server_addrs = [ 10.0.0.241:6379, ] status = Up description = Not using Kafka for UVE Aggregation >>, << type = Redis-UVE name = 10.0.0.241:6379 server_addrs = [ 10.0.0.241:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = OpServer server_addrs = [ 10.0.0.241:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Redis-UVE:10.0.0.241:6379[Redis Instance initialized], Zookeeper:OpServer[Connection to Zookeeper initialized], Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 05/01/2025 02:07:23.503 7f7bd1502d68 [contrail-analytics-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.503 7f7bd1502d68 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.503 7f7bd1502d68 [contrail-analytics-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3254-1.:Analytics:contrail-analytics-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8090 start_time = 1746065243501645 collector_name = collector_ip = collector_list = [ 10.0.0.241:8086, ] >> sm_queue_count = 5 max_sm_queue_count = 5 >> 05/01/2025 02:07:23.506 7f7bd1502d68 [contrail-analytics-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.508 7f7bd6e51c18 [contrail-analytics-api] [INFO]: Analytics Discovery zk start 05/01/2025 02:07:23.517 7f7bd1befeb8 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.517 7f7bd1befeb8 [contrail-analytics-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3254-1. process_status = [ << module_id = contrail-analytics-api instance_id = 0 state = Non-Functional connection_infos = [ << type = UvePartitions name = UVE-Aggregation server_addrs = [ 10.0.0.241:6379, ] status = Up description = Not using Kafka for UVE Aggregation >>, << type = Redis-UVE name = 10.0.0.241:6379 server_addrs = [ 10.0.0.241:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = OpServer server_addrs = [ 10.0.0.241:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.241:8082, ] status = Initializing description = Connection to API Server initialized >>, ] description = Redis-UVE:10.0.0.241:6379[Redis Instance initialized], Zookeeper:OpServer[Connection to Zookeeper initialized], Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[Connection to API Server initialized] connection down >>, ] >> 05/01/2025 02:07:23.517 7f7bd1befeb8 [contrail-analytics-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.520 7f7bd577f240 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.521 7f7bd577f240 [contrail-analytics-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3254-1. process_status = [ << module_id = contrail-analytics-api instance_id = 0 state = Non-Functional connection_infos = [ << type = UvePartitions name = UVE-Aggregation server_addrs = [ 10.0.0.241:6379, ] status = Up description = Not using Kafka for UVE Aggregation >>, << type = Redis-UVE name = 10.0.0.241:6379 server_addrs = [ 10.0.0.241:6379, ] status = Up >>, << type = Zookeeper name = OpServer server_addrs = [ 10.0.0.241:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.241:8082, ] status = Initializing description = Connection to API Server initialized >>, ] description = Zookeeper:OpServer[Connection to Zookeeper initialized], Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[Connection to API Server initialized] connection down >>, ] >> 05/01/2025 02:07:23.521 7f7bd577f240 [contrail-analytics-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.524 7f7bd8be3b70 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.525 7f7bd8be3b70 [contrail-analytics-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3254-1. process_status = [ << module_id = contrail-analytics-api instance_id = 0 state = Non-Functional connection_infos = [ << type = UvePartitions name = UVE-Aggregation server_addrs = [ 10.0.0.241:6379, ] status = Up description = Not using Kafka for UVE Aggregation >>, << type = Redis-UVE name = 10.0.0.241:6379 server_addrs = [ 10.0.0.241:6379, ] status = Up >>, << type = Zookeeper name = OpServer server_addrs = [ 10.0.0.241:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.241:8082, ] status = Up description = Connection to API Server established >>, ] description = Zookeeper:OpServer[Connection to Zookeeper initialized], Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 05/01/2025 02:07:23.525 7f7bd8be3b70 [contrail-analytics-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.525 7f7bd57e48d0 [contrail-analytics-api] [INFO]: Analytics Discovery listen CONNECTED 05/01/2025 02:07:23.525 7f7bd57e48d0 [contrail-analytics-api] [INFO]: Analytics Discovery to publish None 05/01/2025 02:07:23.525 7f7bd91748d0 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.529 7f7bd91748d0 [contrail-analytics-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3254-1. process_status = [ << module_id = contrail-analytics-api instance_id = 0 state = Non-Functional connection_infos = [ << type = UvePartitions name = UVE-Aggregation server_addrs = [ 10.0.0.241:6379, ] status = Up description = Not using Kafka for UVE Aggregation >>, << type = Redis-UVE name = 10.0.0.241:6379 server_addrs = [ 10.0.0.241:6379, ] status = Up >>, << type = Zookeeper name = OpServer server_addrs = [ 10.0.0.241:2181, ] status = Up description = Connection to Zookeeper re-established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.241:8082, ] status = Up description = Connection to API Server established >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 05/01/2025 02:07:23.529 7f7bd91748d0 [contrail-analytics-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.529 7f7bd91748d0 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.529 7f7bd91748d0 [contrail-analytics-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3254-1. process_status = [ << module_id = contrail-analytics-api instance_id = 0 state = Non-Functional connection_infos = [ << type = UvePartitions name = UVE-Aggregation server_addrs = [ 10.0.0.241:6379, ] status = Up description = Not using Kafka for UVE Aggregation >>, << type = Redis-UVE name = 10.0.0.241:6379 server_addrs = [ 10.0.0.241:6379, ] status = Up >>, << type = Zookeeper name = OpServer server_addrs = [ 10.0.0.241:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.241:8082, ] status = Up description = Connection to API Server established >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 05/01/2025 02:07:23.529 7f7bd91748d0 [contrail-analytics-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 05/01/2025 02:07:23.535 7f7bd6e51c18 [contrail-analytics-api] [INFO]: Analytics Discovery Watcher AlarmGenerator Children [] 05/01/2025 02:07:23.537 7f7bd6e51c18 [contrail-analytics-api] [INFO]: Analytics Discovery OpServer reconnect 05/01/2025 02:07:23.539 7f7bd6e51c18 [contrail-analytics-api] [INFO]: ensure /analytics-discovery-/OpServer 05/01/2025 02:07:23.539 7f7bd6e51c18 [contrail-analytics-api] [INFO]: zk state CONNECTED (CONNECTED) 05/01/2025 02:07:23.545 7f7bd6e51c18 [contrail-analytics-api] [INFO]: check for /analytics-discovery-/OpServer/cn-jenkins-deploy-platform-ansible-os-3254-1.-0 05/01/2025 02:07:27.506 7f7bd57ad8d0 [contrail-analytics-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 05/01/2025 02:07:27.507 7f7bd57ad8d0 [contrail-analytics-api] [INFO]: Session Event: TCP Connected 05/01/2025 02:07:27.507 7f7bd57ad8d0 [contrail-analytics-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 05/01/2025 02:07:27.507 7f7bd57ad8d0 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 05/01/2025 02:07:27.507 7f7bd57ad8d0 [contrail-analytics-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3254-1. process_status = [ << module_id = contrail-analytics-api instance_id = 0 state = Non-Functional connection_infos = [ << type = UvePartitions name = UVE-Aggregation server_addrs = [ 10.0.0.241:6379, ] status = Up description = Not using Kafka for UVE Aggregation >>, << type = Redis-UVE name = 10.0.0.241:6379 server_addrs = [ 10.0.0.241:6379, ] status = Up >>, << type = Zookeeper name = OpServer server_addrs = [ 10.0.0.241:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ 10.0.0.241:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.241:8082, ] status = Up description = Connection to API Server established >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 05/01/2025 02:07:27.507 7f7bd57ad8d0 [contrail-analytics-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 05/01/2025 02:07:27.507 7f7bd57ad8d0 [contrail-analytics-api] [INFO]: Processing event[EvTcpConnected] in state[Connect] 05/01/2025 02:07:27.507 7f7bd57ad8d0 [contrail-analytics-api] [DEBUG]: Send sandesh control message. uve type count # 7 05/01/2025 02:07:27.508 7f7bd57ad8d0 [contrail-analytics-api] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 05/01/2025 02:07:27.508 7f7bd57ad8d0 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 05/01/2025 02:07:27.508 7f7bd57ad8d0 [contrail-analytics-api] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 05/01/2025 02:07:27.508 7f7bd57ad390 [contrail-analytics-api] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 05/01/2025 02:07:27.509 7f7bd57ad390 [contrail-analytics-api] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 05/01/2025 02:07:27.511 7f7bd1502ac8 [contrail-analytics-api] [DEBUG]: Extracted sandesh length: 0000001017 05/01/2025 02:07:27.511 7f7bd1502ac8 [contrail-analytics-api] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 05/01/2025 02:07:27.511 7f7bd097b978 [contrail-analytics-api] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 05/01/2025 02:07:27.511 7f7bd097b978 [contrail-analytics-api] [DEBUG]: Number of uve types in sandesh control message is 0 05/01/2025 02:07:27.512 7f7bd097b978 [contrail-analytics-api] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:AnalyticsApiInfoUVE] map 05/01/2025 02:07:27.512 7f7bd097b978 [contrail-analytics-api] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 05/01/2025 02:07:27.512 7f7bd097b978 [contrail-analytics-api] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 05/01/2025 02:07:27.512 7f7bd097b978 [contrail-analytics-api] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 05/01/2025 02:07:27.512 7f7bd57ad390 [contrail-analytics-api] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 05/01/2025 02:07:27.513 7f7bd57ad390 [contrail-analytics-api] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 05/01/2025 02:07:34.023 7f7bd097b978 [contrail-analytics-api] [DEBUG]: uve_type:vrouter user_token:gAAAAABoEtdlHFKRyiA6rdARqgPFnDemNjwA_x-sThrJiGVaO5W_VQbVHVycZ_RMAxyLtUS52qVVLNDxrSQH9pSFVt1q5KxGEbWuVEHegViNoPNTs7CDXZ4_GImr-mDcnlBwPRA8hI0Dp1K_CwffOa9WD_jJOYi7sNciYz8fFcynNdNTpBMdRcQ 05/01/2025 02:07:34.023 7f7bd097b978 [contrail-analytics-api] [DEBUG]: user resources are None 05/01/2025 02:07:34.025 7f7bd57ad0f0 [contrail-analytics-api] [INFO]: __default__ [SYS_INFO]: AnalyticsApiStats: api_stats = << operation_type = GET remote_ip = 10.0.0.241 object_type = vrouter request_url = http://10.0.0.241:8081/analytics/uves/vrouter/%2A?cfilt=VrouterAgent:mode response_time_in_usec = 1185 response_size_objects = 0 response_size_bytes = 0 resp_code = 200 useragent = python-requests/2.27.1 node = cn-jenkins-deploy-platform-ansible-os-3254-1. >> 05/01/2025 02:07:34.363 7f7bd097b978 [contrail-analytics-api] [DEBUG]: uve_type:vrouter user_token:gAAAAABoEtdm5l31jdhHCXZXOL5Uqvu_TEpq-LcCAO0qGpHybSDxnRCDg10oVlcSf7iJcHfX0Tt_NhYnydAxVRbD1Xe07RiuWuC2A11K5CF8dhHTZa2kgoiCaj8U52JU2Z8irhU-76kTtKxGGkKW_8GpdhG29ZM5mEO02918LFtRVN_MqYXTjGI 05/01/2025 02:07:34.364 7f7bd097b978 [contrail-analytics-api] [DEBUG]: user resources are None 05/01/2025 02:07:34.365 7f7bd1bea630 [contrail-analytics-api] [INFO]: __default__ [SYS_INFO]: AnalyticsApiStats: api_stats = << operation_type = GET remote_ip = 10.0.0.241 object_type = vrouter request_url = http://10.0.0.241:8081/analytics/uves/vrouter/%2A?cfilt=NodeStatus:process_status response_time_in_usec = 1575 response_size_objects = 0 response_size_bytes = 0 resp_code = 200 useragent = python-requests/2.27.1 node = cn-jenkins-deploy-platform-ansible-os-3254-1. >> 05/01/2025 02:07:35.097 7f7bd1502ac8 [contrail-analytics-api] [DEBUG]: Extracted sandesh length: 0000001436 05/01/2025 02:07:35.097 7f7bd1502ac8 [contrail-analytics-api] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 05/01/2025 02:07:35.097 7f7bd8be3b70 [contrail-analytics-api] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 05/01/2025 02:07:35.097 7f7bd8be3b70 [contrail-analytics-api] [DEBUG]: Number of uve types in sandesh control message is 3 05/01/2025 02:07:35.098 7f7bd8be3b70 [contrail-analytics-api] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:AnalyticsApiInfoUVE] map 05/01/2025 02:07:35.098 7f7bd8be3b70 [contrail-analytics-api] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 05/01/2025 02:07:35.098 7f7bd8be3b70 [contrail-analytics-api] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 05/01/2025 02:07:35.098 7f7bd1bea630 [contrail-analytics-api] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 05/01/2025 02:07:35.098 7f7bd1bea630 [contrail-analytics-api] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 05/01/2025 02:07:37.447 7f7bd57e4780 [contrail-analytics-api] [INFO]: Analytics Discovery Watcher AlarmGenerator Children ['cn-jenkins-deploy-platform-ansible-os-3254-1.-0'] 05/01/2025 02:07:43.550 7f7bd6e51c18 [contrail-analytics-api] [INFO]: Analytics Discovery OpServer reconnect 05/01/2025 02:07:43.551 7f7bd6e51c18 [contrail-analytics-api] [INFO]: ensure /analytics-discovery-/OpServer 05/01/2025 02:07:43.551 7f7bd6e51c18 [contrail-analytics-api] [INFO]: zk state CONNECTED (CONNECTED) 05/01/2025 02:07:43.552 7f7bd6e51c18 [contrail-analytics-api] [INFO]: check for /analytics-discovery-/OpServer/cn-jenkins-deploy-platform-ansible-os-3254-1.-0 05/01/2025 02:07:43.556 7f7bd6e51c18 [contrail-analytics-api] [INFO]: Analytics Discovery AlarmGenerator ChildData : child cn-jenkins-deploy-platform-ansible-os-3254-1.-0, data b'{"ip-address": "10.0.0.241", "instance-id": "0", "redis-ip": "10.0.0.241", "redis-port": "6379", "redis-agg-db": 7, "partitions": "{}"}', event None 05/01/2025 02:07:43.557 7f7bd6e51c18 [contrail-analytics-api] [INFO]: Analytics Discovery AlarmGenerator ChildData : child cn-jenkins-deploy-platform-ansible-os-3254-1.-0, data OrderedDict([('instance-id', '0'), ('ip-address', '10.0.0.241'), ('partitions', '{}'), ('redis-agg-db', 7), ('redis-ip', '10.0.0.241'), ('redis-port', '6379')]), event GET 05/01/2025 02:07:46.542 7f7bd57e4780 [contrail-analytics-api] [INFO]: Analytics Discovery AlarmGenerator ChildData : child cn-jenkins-deploy-platform-ansible-os-3254-1.-0, data b'{"ip-address": "10.0.0.241", "instance-id": "0", "redis-ip": "10.0.0.241", "redis-port": "6379", "redis-agg-db": 7, "partitions": "{}"}', event WatchedEvent(type='CHANGED', state='CONNECTED', path='/analytics-discovery-/AlarmGenerator/cn-jenkins-deploy-platform-ansible-os-3254-1.-0') 05/01/2025 02:07:47.548 7f7bd57e4780 [contrail-analytics-api] [INFO]: Analytics Discovery AlarmGenerator ChildData : child cn-jenkins-deploy-platform-ansible-os-3254-1.-0, data b'{"ip-address": "10.0.0.241", "instance-id": "0", "redis-ip": "10.0.0.241", "redis-port": "6379", "redis-agg-db": 7, "partitions": "{\\"0\\": 1746065266780491, \\"1\\": 1746065266791667, \\"2\\": 1746065266809669, \\"3\\": 1746065266833125, \\"4\\": 1746065266848279, \\"5\\": 1746065266864385, \\"6\\": 1746065266890285, \\"8\\": 1746065266898324, \\"7\\": 1746065266898513, \\"9\\": 1746065266913755, \\"10\\": 1746065266935727, \\"11\\": 1746065266977145, \\"12\\": 1746065266997725, \\"13\\": 1746065267003070, \\"14\\": 1746065267026527, \\"15\\": 1746065267048112, \\"16\\": 1746065267064586, \\"17\\": 1746065267117246, \\"18\\": 1746065267136455, \\"19\\": 1746065267159284, \\"20\\": 1746065267177407, \\"21\\": 1746065267200663, \\"22\\": 1746065267229634, \\"23\\": 1746065267247471, \\"24\\": 1746065267293276, \\"25\\": 1746065267312510, \\"26\\": 1746065267360713, \\"27\\": 1746065267371770, \\"28\\": 1746065267396075, \\"29\\": 1746065267415861}"}', event WatchedEvent(type='CHANGED', state='CONNECTED', path='/analytics-discovery-/AlarmGenerator/cn-jenkins-deploy-platform-ansible-os-3254-1.-0') 05/01/2025 02:08:34.693 7f7bd57acb70 [contrail-analytics-api] [DEBUG]: uve_type:vrouter user_token:gAAAAABoEteiV-Duyka3vKOFZsp3eBBQVK6Ruy7spvsZGT5A42vt3t1b5knoPVc30PB91tqrilZ2_rn_9sY2LOtxVZhg2Isb6e9Db51R-mwloo4UhsjitgY9AyxYpcRi49G0O6lmssHK7sygRyUioiTk1VRnOpX2YqOSq7RjIfOmUighceQ1oTo 05/01/2025 02:08:34.693 7f7bd57acb70 [contrail-analytics-api] [DEBUG]: user resources are None 05/01/2025 02:08:34.694 7f7bd57ad240 [contrail-analytics-api] [INFO]: __default__ [SYS_INFO]: AnalyticsApiStats: api_stats = << operation_type = GET remote_ip = 10.0.0.241 object_type = vrouter request_url = http://10.0.0.241:8081/analytics/uves/vrouter/%2A?cfilt=VrouterAgent:mode response_time_in_usec = 1094 response_size_objects = 0 response_size_bytes = 0 resp_code = 200 useragent = python-requests/2.27.1 node = cn-jenkins-deploy-platform-ansible-os-3254-1. >> 05/01/2025 02:08:35.012 7f7bd57acb70 [contrail-analytics-api] [DEBUG]: uve_type:vrouter user_token:gAAAAABoEteiOCPtbGzvjZo7gYbZoYicXYgku0KsZfsTapF8srO3qgnC6EWKb36j4FyxOEFOTiZmxhiJz2ito0suHU1ekWqJIDV1zOhCN34iuER5gn1Doa9iGVdQdz1PFP50EbVmVFNLY9q2DO9y-I7wUyJQo87RjLzPYS9qfzi6TohyNfZRqYU 05/01/2025 02:08:35.012 7f7bd57acb70 [contrail-analytics-api] [DEBUG]: user resources are None 05/01/2025 02:08:35.013 7f7bd1be90f0 [contrail-analytics-api] [INFO]: __default__ [SYS_INFO]: AnalyticsApiStats: api_stats = << operation_type = GET remote_ip = 10.0.0.241 object_type = vrouter request_url = http://10.0.0.241:8081/analytics/uves/vrouter/%2A?cfilt=NodeStatus:process_status response_time_in_usec = 823 response_size_objects = 0 response_size_bytes = 0 resp_code = 200 useragent = python-requests/2.27.1 node = cn-jenkins-deploy-platform-ansible-os-3254-1. >> 05/01/2025 02:09:15.460 7f7bd57acb70 [contrail-analytics-api] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 05/01/2025 02:09:17.622 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 05/01/2025 02:09:35.323 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: uve_type:vrouter user_token:gAAAAABoEtffUjQI4NviWr32LH3OmQ7AXRW-AIzOZmsc6Ibc5HyoikX9t7WEcoRtmwrWni-WKSZsGqi0ZlXRG_2ZznAd-1CW2U9T77W11xa-QWHA6qHGL7BP2J8-h88vyDmOHCNylFvJlbGvMeFYnNFaOpSb--_w3wbWwgPPgqiqh3p48doQNjg 05/01/2025 02:09:35.323 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: user resources are None 05/01/2025 02:09:35.324 7f7bd91a8630 [contrail-analytics-api] [INFO]: __default__ [SYS_INFO]: AnalyticsApiStats: api_stats = << operation_type = GET remote_ip = 10.0.0.241 object_type = vrouter request_url = http://10.0.0.241:8081/analytics/uves/vrouter/%2A?cfilt=VrouterAgent:mode response_time_in_usec = 911 response_size_objects = 0 response_size_bytes = 0 resp_code = 200 useragent = python-requests/2.27.1 node = cn-jenkins-deploy-platform-ansible-os-3254-1. >> 05/01/2025 02:09:35.645 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: uve_type:vrouter user_token:gAAAAABoEtffTH45S2-iDgtsZmBjDU0-OAY404CLFbLc80LIJGhYmTk4E4wFU-N5bKyCa4gWka_W8CakkgpNw9DYdQCuenacddvGF-eBG8ItTFjRea9NgtZ6ypvj6Fv2Qi5sSZfrjPcKYe_W8N4erGWRRu7wm4Nk-xHWwMfihLRoJ7ljY-qBBH4 05/01/2025 02:09:35.645 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: user resources are None 05/01/2025 02:09:35.648 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: Computed ObjectVRouter:cn-jenkins-deploy-platform-ansible-os-3254-1 as ['NodeStatus'] 05/01/2025 02:09:35.649 7f7bd57df780 [contrail-analytics-api] [INFO]: __default__ [SYS_INFO]: AnalyticsApiStats: api_stats = << operation_type = GET remote_ip = 10.0.0.241 object_type = vrouter request_url = http://10.0.0.241:8081/analytics/uves/vrouter/%2A?cfilt=NodeStatus:process_status response_time_in_usec = 3259 response_size_objects = 1 response_size_bytes = 932 resp_code = 200 useragent = python-requests/2.27.1 node = cn-jenkins-deploy-platform-ansible-os-3254-1. >> 05/01/2025 02:10:36.324 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: uve_type:vrouter user_token:gAAAAABoEtgc0hFahP0R3ofjpZBm-d86lvss9VHx3VL3TzcALBssAbNcvDOMm4OIzFkRzwyPiGnflG2rrRs9wIpJ5Zc7TxgmvzlBtIx3FzKSXeFtqUp84iCuBg4bLs6qbES6bR5OPFv2Ye0qg1j8a4n4wYz1CFLTYg_c0s-lAmv382NcvEwPSmQ 05/01/2025 02:10:36.324 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: user resources are None 05/01/2025 02:10:36.327 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: Computed ObjectVRouter:cn-jenkins-deploy-platform-ansible-os-3254-1 as ['VrouterAgent'] 05/01/2025 02:10:36.328 7f7bd57e4eb8 [contrail-analytics-api] [INFO]: __default__ [SYS_INFO]: AnalyticsApiStats: api_stats = << operation_type = GET remote_ip = 10.0.0.241 object_type = vrouter request_url = http://10.0.0.241:8081/analytics/uves/vrouter/%2A?cfilt=VrouterAgent:mode response_time_in_usec = 3108 response_size_objects = 1 response_size_bytes = 104 resp_code = 200 useragent = python-requests/2.27.1 node = cn-jenkins-deploy-platform-ansible-os-3254-1. >> 05/01/2025 02:10:37.009 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: uve_type:vrouter user_token:gAAAAABoEtgcbuqGPMYzIS7lpnFEaKi8Mhqmgjehs_7PeGsjEKdVfKp861_6fpdQr-NlgD5AFhZrKnZblk7mBWG7FHyzzgYYmqWP0dxwfjlCSMoZyt7RcDfB8KapgXGALOLR-6-EWL0pxpqicElPdDcLgLp6JBpqrmbweIExpPGBLxayhB-7s3g 05/01/2025 02:10:37.009 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: user resources are None 05/01/2025 02:10:37.013 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: Computed ObjectVRouter:cn-jenkins-deploy-platform-ansible-os-3254-1 as ['NodeStatus'] 05/01/2025 02:10:37.013 7f7bd57acb70 [contrail-analytics-api] [INFO]: __default__ [SYS_INFO]: AnalyticsApiStats: api_stats = << operation_type = GET remote_ip = 10.0.0.241 object_type = vrouter request_url = http://10.0.0.241:8081/analytics/uves/vrouter/%2A?cfilt=NodeStatus:process_status response_time_in_usec = 3720 response_size_objects = 1 response_size_bytes = 932 resp_code = 200 useragent = python-requests/2.27.1 node = cn-jenkins-deploy-platform-ansible-os-3254-1. >> 05/01/2025 02:10:45.771 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: resource list for bgp-peer is None 05/01/2025 02:10:46.694 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: resource list for bgp-peer is None 05/01/2025 02:11:37.648 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: uve_type:vrouter user_token:gAAAAABoEthZJddG_TvwZVjOHo36_hqeiSPGc8kctGttmWW0xGxQ17punaBvYQSRq9NG6Wqt3pxzbDqnMG29-YPeuGyXpRUofgSIU3D0cbISgHS0VKb5osNsYGtBK-XYFlusilbBm0YMKtGAx5F1U8Jxm3G_H42N6oUIcYfgdxCsi7hMf4mi_hA 05/01/2025 02:11:37.648 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: user resources are None 05/01/2025 02:11:37.650 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: Computed ObjectVRouter:cn-jenkins-deploy-platform-ansible-os-3254-1 as ['VrouterAgent'] 05/01/2025 02:11:37.650 7f7bd57ad0f0 [contrail-analytics-api] [INFO]: __default__ [SYS_INFO]: AnalyticsApiStats: api_stats = << operation_type = GET remote_ip = 10.0.0.241 object_type = vrouter request_url = http://10.0.0.241:8081/analytics/uves/vrouter/%2A?cfilt=VrouterAgent:mode response_time_in_usec = 2126 response_size_objects = 1 response_size_bytes = 104 resp_code = 200 useragent = python-requests/2.27.1 node = cn-jenkins-deploy-platform-ansible-os-3254-1. >> 05/01/2025 02:11:38.003 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: uve_type:vrouter user_token:gAAAAABoEthZLdLOfX0F_kJdgcY7jXjZwNkgJGch4MQs3uMbvTMi49qRsSCoWmKCOncB_2GzOpECTdFujoECfhTzCLmInfh4z5xzfI4cbPpjopCJzujMc83KRCIdIlqQkwFaBxnfp-x8dT2kWgllpOfi_ih1Wk4RiZkEvveDy6rYhbBh6OQXJmA 05/01/2025 02:11:38.003 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: user resources are None 05/01/2025 02:11:38.005 7f7bd42e7240 [contrail-analytics-api] [DEBUG]: Computed ObjectVRouter:cn-jenkins-deploy-platform-ansible-os-3254-1 as ['NodeStatus'] 05/01/2025 02:11:38.006 7f7bd57a9ac8 [contrail-analytics-api] [INFO]: __default__ [SYS_INFO]: AnalyticsApiStats: api_stats = << operation_type = GET remote_ip = 10.0.0.241 object_type = vrouter request_url = http://10.0.0.241:8081/analytics/uves/vrouter/%2A?cfilt=NodeStatus:process_status response_time_in_usec = 2695 response_size_objects = 1 response_size_bytes = 932 resp_code = 200 useragent = python-requests/2.27.1 node = cn-jenkins-deploy-platform-ansible-os-3254-1. >> 05/01/2025 02:12:40.401 7f7bd91a0240 [contrail-analytics-api] [DEBUG]: uve_type:vrouter user_token:gAAAAABoEtiYYkaiyCXKCKSl3THUSHISRRymHmSZ0rqGDVM_utLrQ1VYKkqwg62YaxPvaK7RxGEaCnlK8CX3FbjPntSLusgn-S3Sj_Rcqttpt_KKOC1ELZ5tvOdOHzQ4Vv96h4c24d-WtQ6AXyJTMM1fKvSlCJ8HhmcaMC1UGPqg-M7azd3IQqE 05/01/2025 02:12:40.401 7f7bd91a0240 [contrail-analytics-api] [DEBUG]: user resources are None 05/01/2025 02:12:40.403 7f7bd91a0240 [contrail-analytics-api] [DEBUG]: Computed ObjectVRouter:cn-jenkins-deploy-platform-ansible-os-3254-1 as ['VrouterAgent'] 05/01/2025 02:12:40.404 7f7bd6f368d0 [contrail-analytics-api] [INFO]: __default__ [SYS_INFO]: AnalyticsApiStats: api_stats = << operation_type = GET remote_ip = 10.0.0.241 object_type = vrouter request_url = http://10.0.0.241:8081/analytics/uves/vrouter/%2A?cfilt=VrouterAgent:mode response_time_in_usec = 2732 response_size_objects = 1 response_size_bytes = 104 resp_code = 200 useragent = python-requests/2.27.1 node = cn-jenkins-deploy-platform-ansible-os-3254-1. >> 05/01/2025 02:12:40.720 7f7bd91a0240 [contrail-analytics-api] [DEBUG]: uve_type:vrouter user_token:gAAAAABoEtiYvyOBIh73qtvXurbKIHHxje5lQhZuvYPXqdirqTAXY3IFo5cz49Wpyd2HRobwx5xwgpScR6bXWHaB1ausUozEMGcbQT9iHNZqdHosoFIwNWWXNMzbXEMSVBEQPZ3Xg94ToasBEKUlZLxB3qJffvnqI1-bblYL1qlglTaY-4w5yIs 05/01/2025 02:12:40.721 7f7bd91a0240 [contrail-analytics-api] [DEBUG]: user resources are None 05/01/2025 02:12:40.724 7f7bd91a0240 [contrail-analytics-api] [DEBUG]: Computed ObjectVRouter:cn-jenkins-deploy-platform-ansible-os-3254-1 as ['NodeStatus'] 05/01/2025 02:12:40.725 7f7bd57db828 [contrail-analytics-api] [INFO]: __default__ [SYS_INFO]: AnalyticsApiStats: api_stats = << operation_type = GET remote_ip = 10.0.0.241 object_type = vrouter request_url = http://10.0.0.241:8081/analytics/uves/vrouter/%2A?cfilt=NodeStatus:process_status response_time_in_usec = 3942 response_size_objects = 1 response_size_bytes = 932 resp_code = 200 useragent = python-requests/2.27.1 node = cn-jenkins-deploy-platform-ansible-os-3254-1. >> 05/01/2025 02:12:46.661 7f7bd91a0240 [contrail-analytics-api] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map